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Foreword 



In choosing the theme for this year’s Annual Meeting, we expected that the first phrase of the Commission's new 
mission Statement-Serving the Common Good-would provide an exciting focus for many presenters. And it did. 
In this set of papers we read about efforts of colleges to enhance access to higher learning for many types of 
students. We learn of important efforts to breach outdated walls that make difficult the transportability of higher 
learning. We can only admire the commitments of many institutions to hold themselves accountable for the 
learning their students achieve. We must applaud the renewed commitment of educators to ensure that higher 
education is as effective in challenging students to be independent and creative thinkers as it is in making them 
employable. 

It is in the new designs that we come to appreciate how different higher education must become from what it 
was barely a decade or two ago. For example, you find in these pages multiple references to portfolios, not simply 
as a tool for measuring student learning but also as a method for institutions to make public their goals and their 
achievements. Technology shapes the tools institutions use to evaluate institutional effectiveness even as it 
reconfigures the teaching and learning environments within and among institutions. And while quality 
improvement is not new to higher educational institutions, its importance in creating new designs for 
accreditation is. 

This Annual Meeting is unique because the core business of the Commission is “assuring and advancing the 
quality of higher learning.” But the basic culture of higher education is shifting, with increasing emphasis on 
accountability for student learning. Many presenters take current practices and give them new power in these 
changing times: self-study, strategic planning, mission revision. Several excellent papers speak to the central 
challenge of keeping vital the role of the total faculty in protecting protect academic quality in this period of 
change in the academic culture. 

Another Annual Meeting with The Collection of Papers becoming another major resource forthe Commission and 
all of its affiliated institutions. The lessons contained in this volume are many and varied. They deserve our study 
and use. 



Steven D. Crow 
Executive Director 
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March 1, 2001 



Preface 



On behalf of the Commission, I am pleased to present the 2001 edition of the Collection of Papers on Self-Study 
and Institutional Improvement. Now in its seventeenth year, the Collection of Papers has moved beyond a 
supplement to the Meeting presentations to be a remarkable resource throughout the year for all who are 
interested in issues of higher education quality. We are grateful to our speakers for their generous contributions 
to the work of the Commission through these papers as well as through their presentations at the Annual Meeting. 

The theme of the 2001 Annual Meeting, "Serving the Common Good: New Designs in Higher Education,” is woven 
throughout the Collection, with collaboration emerging as a strong subtheme in many papers, regardless of the 
focus. For the twelve years of the Commission’s Assessment Initiative, the Collection of Papers has highlighted 
institutional efforts to assess student academic achievement. In this year’s edition, discussions of assessment 
efforts are not limited to the chapters on assessment, but appear in more than forty papers throughoutthe volume. 
In addition, this year’s Collection includes a group of papers from the some of the first institutions participating 
in the Commission’s Academic Quality Improvement Project [AQIP). For those involved in self-study, the 
Collection of Papers goes beyond the policies and procedures provided in the Handbook of Accreditation, to give 
practical advice based on actual experience. Several papers explore alternatives to traditional self-study. 

Producing a book of this size in five weeks requires significant team effort. Special thanks are given to the 
following individuals who made the 2001 Collection possible: Marisol Gomez and Viki Berberich, for their help 
in processing initial submissions and preparing files; Sybil Sosin, for her valuable editorial assistance; Gerald Van 
Kollenburg, for his extraordinary assistance with the layout, particularly the charts and graphics; Kathleen 
Herring, for the beautiful cover design; and Aaron Marsh of Honi Graphics, for always getting the book printed 
in time for the Meeting. 

The Commission invites your comments about the Collection of Papers and welcomes your suggestions for future 
topics for the Annual Meeting program. I hope that you will consider participation as a speaker at a future 
Meeting. The strength of the Annual Meeting lies in the willingness of our institutions to share their experiences 
with others. I look forward to seeing you in at the Meeting. 



Susan E. Van Kollenburg 
Editor 

Associate Director for Programs, Publications, 
and Member Services 



March 1,2001 




Ed. note: The name of the Commission was changed from the Commission on Institutions of Higher Education to 
The Higher Learning Commission effective January 1, 2001. In their papers, authors may have referred to this 
organization as the North Central Association, the Commission on Institutions of Higher Education, or The Higher 
Learning Commission. Information about the name change is available on the Commission’s web site: 
www.ncahigherlearningcommission.org. 
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Chapter l New Designs in Higher Education. Improving Student Learning: Selected PEW Projects/ 3 



Electronic Institutional Portfolios: 
Using the Web for Assessment, 
Improvement, and Accountability 



Susan Kahn 
Ann Feldman 
Barbara Ellen Walvoord 



Overview 

Electronic institutional portfolios are an emerging medium that uses the World Wide Web to exhibit, assess, and report 
on teaching and learning at the institutional level. In the Urban Universities Portfolio Project (UUPP), a two-and-a- 
half-year-old national initiative, the portfolios underdevelopment go beyond traditional presentational models for 
reporting assessment results in several ways: they use the electronic environment to engage groups within the 
university in collective self-examination and reflection on student learning and educational effectiveness; they 
incorporate interactivity with portfolio users; and they employ multimedia to actually show-not just describe or 
explain-teaching and learning. 

In addition to capturing student learning in more vivid and varied ways than a paper report might do, electronic 
institutional portfolios can be especially effective for demonstrating continuous institutional improvement and for 
evaluating an institution within the framework of its own particular mission. These strengths are consistent with current 
emphases in accreditation; regional accrediting associations may thus find electronic institutional portfolios useful as 
a new form of self-study. Indeed, one association, the Western Association of Schools and Colleges, recently mandated 
institutional portfolios in place of traditional self-studies, and several of the other regionals may follow suit The 
description below of how one institution is documenting the teaching and learning of writing illustrates the utility of 
portfolios for presenting the kinds of evidenceof educational effectiveness that accreditors and other stakeholders seek. 



The Urban Universities Portfolio Project 

The Urban Universities Portfolio Project: Assuring Quality for Multiple Publics brings together six leading urban public 
universities' to develop electronic institutional portfolios that demonstrate the universities’ effectiveness to various 
groups of stakeholders, including accreditors. Funded by the Pew Charitable Trusts and cosponsored by the American 
Association for Higher Education fAAHE), the project has three main purposes: to enhance internal and external 
stakeholders’ understanding of the mission of urban public universities; to develop a new approach to cultivating 
ongoing internal improvement; and to experiment with new ways of demonstrating and evaluating effectiveness and 
accountability in the context of mission. The vehicle for achieving these purposes is the set of six electronic 
institutional portfolios that project universities are developing. 

Institutional portfolios might be thought of as offspring or siblings of faculty teaching and course portfolios and of 
student learning portfolios. Over the past decade, such portfolios have come into widespread use within higher 
education as vehicles for individual and institutional learning, evaluation, reflection, and improvement. Among the 
key characteristics of these types of portfolios are their inclusion of authentic primary materials as evidence for 
effective teaching and learning, and their use of reflective narrative to provide a context of self-evaluation and 
improvement of faculty and student efforts. These portfolios are generally framed by statements of educational 
philosophy or desired educational outcomes, against which the evidence of actual achievement in the portfolios can 
be measured. 
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Enter the World Wide Web, which vastly expands the possibilities for exhibiting authentic evidence [e.g., graphical 
materials, audio and video of class sessions or of faculty and students discussing teaching and learning experiences], 
for making information easily accessible, and for permitting viewers to examine materials at varying levels of detail. 
Both faculty and students took the opportunity to adapt the portfolio concept to a web format, with some institutions 
requiring or encouraging web-based, rather than paper, portfolios for students. 

The web, with its capacity for storing large amounts of material and enabling multiple pathways through those 
materials, also made feasible the expansion of the portfolio concept to the entire institution. Like faculty and student 
portfolios, institutional portfolios have at their core authentic evidence of teaching and learning, and reflective 
narrative. But the evidence and narrative are selected and created as part of a collective— ideally, institution-wide- 
process of examining and reflecting on teaching and learning and their improvement. In the UUPP, participating 
institutions have found that work on their web-based electronic portfolios has energized and catalyzed assessment 
and planning efforts. All six universities in the project have launched significant assessment and improvement 
initiatives resulting directly from their portfolio work, and have used the portfolio effort to bring greater coherence, 
cohesiveness, and effectiveness to existing improvement initiatives. 

The project envisions portfolio development not as a one-time task, but as an ongoing system that allows a university 
to monitor its performance and document that performance for internal and external stakeholders. The electronic 
portfolio web sites will thus evolve continuously, demonstrating changes and improvements unfolding over real time. 
In this way, the portfolios incorporate a commitment to continuous, rather than episodic, self-assessment and 
improvement and demonstrate the universities’ skills in assessment and self-correction. For accreditation purposes, 
such a standing portfolio might be supplemented by a short set of guidelines for navigating the portfolio site and 
perhaps by an executive summary of portfolio highlights, obviating the need for a massive and expensive self-study 
effort carried out solely for the purposes of an accreditation review. 

As the portfolios develop, the UUPP is experimenting with ways in which site visits and electronic institutional 
portfolios might complement one another, leading to new approaches that external evaluators, especially accreditors, 
might use to learn about and evaluate institutions. Experimental visits in spring 2000 and spring 2001 are using the 
portfolios and other evidence provided by the universities to examine learning outcomes as well as institutions’ 
processes for assuring educational quality and effectiveness. Visitors are considering such questions as: How has the 
institution developed systems for assessing its own performance? What are the standards of evidence? What are the 
results? How can virtual visits to the portfolio web sites be most effectively combined with physical visits to the 
campus? Can visitors combine consultative and evaluative roles? Can they help the institution become a learning 
organization? 



An Institutional Example 

The electronic environment of the World Wide Web presents enormous potential for displaying multiple types of 
qualitative and quantitative evidence for learning. The evolving UUPP portfolios use an array of information-primary 
materials from students and faculty, assessment findings, institutional data, and reflective critiques-to make their 
point about the institutions’ effectiveness in nurturing students’ intellectual development. 

Take, for example, the student writing section of the University of Illinois at Chicago [UIC] portfolio. To demonstrate 
how students learn to write at the university, the portfolio includes information, actual syllabi, and discussion of key 
concepts included in the university’s first-year writing sequence, supplemented by student stories-audio discussion 
by students about their writing experiences at UIC. In addition, the section contains a wealth of information on writing 
assessment: detailed outcomes statements for first-year writing courses, placement criteria and statistics, description 
of writing portfolio assessment criteria, with examples of student work, and results of before-and-after surveys of 
students who have taken required writing courses on their perceptions of gains in specific writing skills, such as 
“writing in a personal voice” or “developing a strong argument.” 

The materials, information, and evidence that UIC presents add up to a detailed characterization of how the first-year 
writing program contributes to students’ undergraduate education. Writing faculty at UIC expect that the portfolio will 
also serve as a resource for the university as it develops upper-level writing requirements. UIC plans to continue 
developing the writing portion of its institutional portfolio, as well as sections on other core learning outcomes that 
will serve both the university’s internal assessment and improvement efforts, and will provide external stakeholders 
with a wide array of evidence of how students acquire skills in writing and other fundamental areas over their 
undergraduate careers. 
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Note 

1. Participating institutions include California State University, Sacramento; Georgia State University; Indiana 
University Purdue University Indianapolis; Portland State University; the University of Illinois at Chicago; and the 
University of Massachusetts Boston. 



Susan Kahn is Director of the Urban Universities Portfolio Project at Indiana University-Purdue University Indianapolis. 
Ann Feldman is Associate Vice Chancellor at the University of Illinois at Chicago. 

Barbara Walvoord is Director of the Kaneb Center for Teaching and Learning at the University of Notre Dame in Indiana. 
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Making Student Learning Central: 
Principles and Practices for 
Implementation 



James Roth 
Mark Gromko 
Susan McGury 
David Wissmann 



Participating Institutions 

| Alverno College, Milwaukee, Wl 
Avila College, Kansas City, MO 
Birmingham Southern College, 
Birmingham, AL 

Bowling Green State University, Bowling 
Green, OH 

California State University-Fullerton 
California State University- Monterey 
Bay, Seaside 

Central Missouri State University, 
Warrensburg 

j Clayton College and State University, 

| Morrow, GA 

I Fort Valley State University, Fort Valley, 
j GA 

| Huston-Tillotson College, Austin, TX 
Indiana University of Pennsylvania, 
Indiana 

Indiana University-Purdue University- 
Indianapolis 

James Madison University, 
Harrisonburg, VA 

Niagara University Niagara, University, 
NY 

i 

North Carolina State University, Raleigh 
Olivet College, Olivet, Ml 
Rivier College, Nashua, NH 
Rose-Hulman Institute of Technology, 
Terre Haute, IN 

Samford University, Birmingham, AL 
The School for New Learning at DePaul 
j University, Chicago, IL 

i Seton Hill College, Greensburg, PA 
State University of New York College at 
| Fredonia 

! Truman State University, Kirksville, MO 
J Unity College, Unity, ME 
! University of Alaska-Southeast, Juneau 
! University of Washington, Seattle 
i University of Wisconsin-LaCrosse 



The Student Learning Outcomes Initiative is a collaborative effort by twenty- 
seven public and private baccalaureate degree-granting institutions to create 
and publish a framework that will identify the assumptions, principles, and 
practices that institutions could use in making student learning the organiz- 
ing principle of their campuses. This initiative is being funded by The Pew 
Charitable Trusts. 



The following is an introduction to the third draft of the document (still in process), 
“Student Learning : A Central Focus for Institutions of Higher Education, " 
by The Student Learning Outcomes Initiative. Publication date: Fall 2001. 



Introduction 

What can we do to improve the quality of student learning in our institutions? 
This is a question that all of us involved in higher education ask ourselves 
regularly. It does not imply that we are not already making significant efforts 
to foster student learning on our campuses. It merely reflects the ongoing 
commitment to making our institutions the most effective learning environ- 
ments we can, and it is the question at the heart of this framework and the 
accompanying institutional examples. The institutions that have participated 
in the authoring of this document are actively engaged in responding to the 
question on our own campuses, and we hope that the document will 
contribute to a public dialogue about howto improve student learning across 
higher education. 

We see this publication as an invitation to inquiry into and discourse about 
what it means to make student learning a central focus for institutions of 
higher education. In this context, the framework we have developed is a way 
ofthinking about student learning ratherthan a blueprint to follow. It reflects 
the kind of intellectual discourse in which we have been engaged during this 
initiative, and we hope it will provide support for productive innovation in the 
service of student learning. 

The reader will notice that we use questions throughout the framework as a 
way of stimulating a particular line of thought or suggesting a specific focus 
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for consideration. This also reflects our own experience in working together to author the framework. The wide variety 
of institutions involved in this collaboration was intentional. We thought this was important in order to represent the 
breadth of institutional cultures in higher education, and this made our discussions rich and complex. We have clearly 
come to agreement on some principles that seem essential to effective student learning, but we also found in our 
discussions no single approach or practice that is appropriate in all settings. We discovered, for example, that even 
arriving at a common language to use in considering issues of teaching, assessment, and learning can be a struggle. 
In this spirit we encourage the readers of this framework to use it as a stimulus for exploration onyourown campuses, 
and across institutions as well. 

We intend the specific examples that accompany the framework to be practical ways to further a student learning focus 
on your campuses. Again, this does not mean that any one of the examples will necessarily transfer directly to your own 
context but they should provide helpful illustrations of how you might address specific concerns at your own institutions. 
Our underlying assumption throughout the framework and the examples is that taking a student learning-centered 
perspective on our work as higher educational institutions can make us more intentional in all that we do. 

In surveying institutions across the country and in our conversations with one another throughout the writing of this 
document, we have found that colleges and universities where student learning is a central focus of faculty, staff, and 
administrators tend to exhibit certain characteristics. We use these following characteristics and related questions 
as the organizing structure of this document. 

0 Achieve clarity about learning outcomes: 

What do we mean by the term, student learning outcomes? 

How do institutions determine learning outcomes? 

How are student learning outcomes related to degree requirements? 

What is involved in conceptualizing student learning developmentally? 

0 Coordinate teaching and assessment to promote student learning: 

How do student learning outcomes assist us to “think pedagogically”? 

What can we learn about our students to assist them in learning? 

What are the relationships between teaching and assessment? 

How can we help students develop processes of self-assessmeni? 

0 Align structures and resources to serve student learning: 

What forms of inquiry foster a focus on student learning? 

How can we support and recognize these forms of inquiry? 

How can we allocate resources to make student learning central? 

How can we take a systemic approach to institutional alignment? 

0 Work continuously to improve the environment for learning: 

What measures are best suited to assess institutional effectiveness? 

How can the results of institutional assessment be used for improvement? 

What is involved in taking collective responsibility for learning? 

How can we encourage innovation in the service of student learning? 

Throughout the framework we explore the meaning of these characteristics, with the questions as both guides and 
food for thought. In the process we offer principles and practices that suggest ways of addressing the characteristics. 
In the section on institutional examples, we illustrate specific approaches that institutions have taken to foster one 
or more of the characteristics. We hope that both the framework and the institutional examples will be of assistance 
in developing sustained arid thoughtful inquiry within and across our institutions. 




James Roth is Professor of History at Alverno College in Milwaukee, Wisconsin. 

Mark Gromko is Vice Provost for Academic Affairs at Bowling Green State University in Bowling Green, Ohio. 

Susan McGury is Assistant Professor of English Literature and Visual Art at The School for New Learning at DePaul 
University in Chicago, Illinois. 

David Wissmann is Professor of Sociology at Avila College in Kansas City, Missouri. 
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Electronic Institutional Portfolios 
and the Self-Study: 

A Dynamic Combination for 
Quality Assurance 



Sharon Hamilton 
Karen Black 



What Is the Issue? 

Accrediting associations across the country are exploring a range of new approaches to assuring quality through the 
accreditation process. Concurrently, institutions of higher learning are also exploring new ways to present evidence 
of quality not only to accrediting associations but also to many other stakeholders. One such innovative approach is 
the Urban Universities Portfolio Project, a Pew-funded, AAHE-sponsored national initiative involving six urban 
campuses in the development of electronic institutional portfolios to demonstrate to multiple audiences how each 
university meets its mission and commitment to its urban constituencies. Indiana University-Purdue University 
Indianapolis (IUPUE), currently preparing for its 2002 NCA accreditation visit, is one of the six universities involved in 
this project It is therefore intensely engaged in developing a functional relationship between traditional self-study 
and the electronic institutional portfolio as documents for quality assurance. 

The electronic portfolio, like the self-study, offers a portrayal of campus mission, strategic plan, goals, processes for 
achieving goals, and indicators of effectiveness in the achievement of goals. Unlike the self-study, however, the 
electronic portfolio is dynamic and interactive, responsive to the interests of visitors, and continually updated. It 
provides a unique blend of quantitative and qualitative evidence, with opportunities to link to and thereby probe more 
deeply into the workings of the university. Moreover, it is not a once-in-ten-years endeavor imposed by the need for 
accreditation, but rather an ongoing demonstration of the effectiveness of the university in achieving its goals. 

One major problem is that most accrediting institutions are not accustomed to dealing with electronic documentation 
and find the traditional self-study more convenient, even as they acknowledge the greater efficiency and greater 
potential of the electronic portfolio. Consequently, while some accrediting agencies, for example the Western 
Association of Schools and Colleges (WASC], are moving toward virtual campus visits based on electronic 
documentation, others prefer the more traditional campus visit and published self-study. This presentation deals with 
that dilemma: how might the electronic institutional portfolio and traditional self-study work together as documen- 
tation without undue duplication or unnecessary use of faculty, staff, and administrative resources of time? 



What Is an Electronic Institutional Portfolio? 



o 
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The electronic institutional portfolio is a new medium for communicating to interested people what we do, how well 
we do it, and what steps we are taking to improve. The prime function of the electronic institutional portfolio is to 
demonstrate to multiple stakeholders the effectiveness of the institution in achieving its mission. It involves, as do most 
portfolios, collection, selection, reflection, and presentation, but in a more fluid, dynamic manner than the typical 
paper-based portfolio. IUPUI, a campus of more than 27,000 students in 21 academic and professional units set in 
the urban center of the Indiana state capital, has a trifold mission: to promote excellence in the areas of student 
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learning; research, scholarship, and creative activity; and civic engagement. While the essential features of this 
mission have not changed over the years, the recent establishment of a community college system in Indiana has 
altered the some of the attributes by which we define excellence in each of these areas. For example, when student 
demographics of the first-year incoming class changed in 2000 from predominately older, part-time students to 
predominately traditional-aged full-time students, the teaching-learning environment changed as well. Furthermore, 
because this demographic change was coupled with a significant change in class standing of the first year students, 
from being predominately in the lower 50 percent of class standing to being predominately in the top 50 percent of 
class standing, expectations for and definitions of excellence in student learning also changed. Similarly, expectations 
for involving students in undergraduate and graduate research changed because more students are on campus full time. 
These are just two of countless examples of how changes in one aspect of a university campus alter the ways in which 
the campus determines its effectiveness in achieving its mission, but they serve to make the point that while ten-year 
self-studies may benchmark changes in how institutional effectiveness is defined, the electronic institutional portfolio 
can provide a more contemporary and timely demonstration of how the institution defines and evaluates its effectiveness. 



How Does the Electronic Portfolio Assure Institutional Quality and Program Effectiveness? 

One of the challenges of the project has been to discover how to maximize the electronic environment to demonstrate 
to a diverse array of stakeholders how effectively IUPUI is achieving its mission. What we have learned is that, once 
campus-wide indicators of effectiveness have been established in relation to student learning, research, and civic 
engagement, the portfolio has the capacity to present both qualitative and quantitative evidence in graphic, narrative, 
and multi-media applications. Visitors to the web site are able to probe as deeply as they need in order to find evidence. 
By developing a conceptual framework with built-in requirements for self-reflection and self-assessment, every item 
in the portfolio becomes a document that supports our claim of striving toward excellence in student learning; 
research, scholarship, and creative activity; and civic engagement. Moreover, the site includes not only institutional 
products, such as analyzed surveys, individual student portfolios, annual reports of schools, and academic programs 
available, but also the processes by which these products come into being. 



How Does the Electronic Institutional Portfolio Relate to and Represent the Work of the 
Campus? 

While the IUPUI portfolio began almost three years ago as part of a three-year funded project, with a team assigned 
to conceptualize and develop the framework of the portfolio, the aim has always been to integrate the portfolio 
completely into the ongoing work of the campus. Consequently, as IUPUI has worked to redefine its expectations for 
student learning; research, scholarship and creative activity; and civic engagement, the portfolio team has been 
included in these conversations and, in some instances, has precipitated them. Hence, there is a dynamic relationship 
between strategic planning for institutional effectiveness; the work of faculty, staff, and students in trying to achieve 
this effectiveness through their disparate roles; and the presentation of both planning and results-processes and 
products, if you will-in the electronic portfolio. 

Much of this session will guide you through the portfolio as it currently exists to show you how it presents both 
quantitative and qualitative evidence of our striving for excellence in relation to our mission. For those unable to attend 
the session, please visit the portfolio at http://www.imir.iupui.edu/iupuifolio. One exciting offspring of this institutional 
portfolio has been the development of individual student portfolios to demonstrate student engagement with six 
foundational principles of undergraduate learning at IUPUI, as well as their progress through their academic majors 
or professional programs. Some of these portfolios are available at http://eport.iupui. The project web site at http:/ 
/www.imir.iupui.edu/portfolio shows how the other five urban institutions involved in the portfolio have developed 
their demonstrations of institutional effectiveness, each in a different way. 

As this brief overview suggests, the development of an electronic institutional portfolio is a formidable task. However, 
it may have a tremendous payoff. Once developed, it may ultimately negate the need for the accreditation self-study 
or may, at the very least, provide supportive resources for a much attenuated and focused self-study. 



What Is the Relationship Between the IUPUI Portfolio and the NCA Self-Study? 

The North Central Association currently prefers a more traditional visit and a published self-study. However, in 2002, 
when North Central will review IUPUI, the interaction between the accreditation process and the IUPUI Electronic 
Portfolio will play out in at least three ways: the portfolio will serve as a repository of documents related to the self- 
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study; the portfolio will serve to explicate assertions made about effectiveness by providing documentation and 
elaboration; and the self-study process and the portfolio development process have a symbiotic relationship wherein 
each serves to guide and be guided by the other. 

As previously mentioned, NCA currently prefers a more traditionally published self-study. However, the entire self- 
study and an index will be part of the portfolio. Thus, the portfolio becomes a repository and historical resource for 
future accreditation processes, whether NCA or discipline-specific. 

To enhance the self-study, the portfolio will serve to explicate, document, and embellish the self-study. The written, 
traditional self-study will refer the reader to the portfolio to further explain or document processes, outcomes, or 
historical events that have taken place on the campus since the last accreditation visit. To accomplish this, the self- 
study groups will use the work already done in forming the portfolio as points of departure. 

Thus, the work of the portfolio influences and, in some ways, guides the work of the self-study. Likewise, the work 
of the self-study committees guides the development of the portfolio. The two processes become interdependent. 
Since the campus-wide Future Group and other planning-related groups guide formal campus planning efforts and 
also serve to guide the accreditation self-study process and the portfolio development, the portfolio and accreditation 
processes thereby become critical to campus planning. 



Sharon Hamilton is Campus Director of the I U PUI Portfolio Project at Indiana University-Purdue University Indianapolis. 

Karen Black is Assistant to the Vice Chancellor for Planning and Institutional Improvement at Indiana University-Purdue 
University Indianapolis. 
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Collaboration and the Common Good: 
New Designs for Developing a 
Unified Approach to Student Learning 



Jack P. Calareso 
Thomas V. Boeke 
Robert A. Bonfiglio 



In recent years, renewed attention both within and outside of the academy has been given to the centrality of student 
learning to the mission of higher education. A critical element in this discussion has been a call for collaboration 
between the academic and student affairs units of our institutions. While providing what some have called a 
“seamless” educational experience for our students may seem to be a call to stray further afield from the classical 
traditions of higher education, the call for collaboration is not based on the intent to diminish the potential for student 
achievement, but to enhance it. The call for collaboration is founded on the ideals of educational excellence and 
institutional integrity, and on the idea that a liberating educational experience is one that contributes to the 
development of the whole person. 

To collaborate is “to work together, especially in a joint intellectual Effort” (Morris, William, ed. The American Heritage 
Dictionary of the English Language. Boston: Houghton Mifflin, 1980, 260). While there may be something romantic 
aboutthe notion of a solitary scholar engaged in painstaking research that sometimes comes to mind when one thinks 
of higher learning, it is more common on our nation's campuses to find teachers working together with students in 
classrooms, labs, and on field excursions, and staff members working together with students in residence halls and 
in clubs and organizations than it is to find the solitary researcher. From the perspective of administering colleges 
and universities, our institutions yearn to be managed collegially. From the sociological perspective, as individuals 
working in shared space, we strive to learn from each other, as the late Ernest Boyer pointed out, in community (Boyer, 
Ernest L. Campus Life: In Search of Community. Princeton, NJ: The Carnegie Foundation for the Advancement of 
Teaching, 1 990). Why is it, then, that we see the need to trumpetthe value of collaboration? Itseems almost as if faculty 
and staff dwell in opposing camps and define collaboration not as working jointly on an intellectual endeavor, but 
as cooperating “treasonably, as with an enemy occupying one’s country” (Morris, 261). 

Part of the dilemma higher education faces has to do not with how we view collaboration, but with how we define 
learning. Conventionally, when one thinks about learning one thinks of the classroom setting, of the distinct roles of 
teacher and student, and of a formal curriculum. Learning, however, takes place in a variety of settings and forms. 
It is the realization that there is educational value in other aspects of the college experience, experiences that go 
beyond the classroom, that prompts one to consider the value of collaboration. 
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This realization has been gaining greater acceptance in American higher education over the course of almost twenty 
years. During this period, the rationale for the perspective that learning encompasses more than classroom learning 
was articulated by the Study Group on the Conditions of Excellence in American Higher Education. In the 
groundbreaking report Involvement in Learning: Realizing the Potential of American Higher Education, it was asserted 
that the degree of student learning and educational effectiveness that takes place in the collegiate setting is directly 
related to the degree to which students are involved in the lives of their institutions beyond the classrooms, engaged 
in such activities as working on campuses, participating in student organizations, and interacting frequently with 
faculty and peers. (Study Group on the Conditions of Excellence in Higher Education. Involvement in Learning : 
Realizing the Potential of American Higher Education. Washington, D.C.: U.S. Department of Education, 1 984.) This 
might seem counterintuitive to the scholar who sees time spent by students in employment and in active involvement 
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in clubs as time better spent reading, writing, and studying. It is a counterintuitive argument if we insist on defining 
learning narrowly. Yet it would seem that by defining learning narrowly we run the risk of betraying the very nature 
of higher education. 

A faculty member’s expertise is rooted in a discipline. However, studies of first-year college students have shown over 
time that rarely is it a specific discipline that attracts students and their parents to higher education. Rather, as 
revealed in national surveys of the freshman year, of Alexander Astin, students primarily choose to come to college 
“to be able to get a better job," “to be able to make more money," and more generally, “to learn more about things” 
(Dey, Eric, Astin, Alexander, and Korn, William. The American Freshman: Twenty-Five Year Trends. Los Angeles: Higher 
Education Research Institute, UCLA, 1991, 112-113.) The collegiate experience is widely valued for its ability to teach 
life skills and for the entree it offers into the world ofthe professions. American higher education is not merely designed 
for postsecondary career training or credentialling. These functions of higher education, while valid in themselves, are 
not the only functions. Our institution’s aims are much loftier, and our expectations are more ambitious. 

In many cases the call to collaboration is one that is readily embraced by the student affairs administrator. It is seen 
by some student affairs personnel as an answer to the perception that they are second-class citizens in the academic 
community. The attraction to faculty is usually less evident. Cross-institutional collaboration should, in fact, be equally 
of interest to presidents, academic administrators, and faculty as well as student affairs’ administrators for, as the 
literature on student learning shows, student involvement with faculty and staff outside of the classroom impacts 
positively on students’ critical thinking ability, intellectual flexibility, capacity for self-awareness and self-reflection, 
ability to communicate effectively, problem-solving skills, conflict-management skills, and time- and stress- 
management skills. (Whitt, Elizabeth J. Student Learning as Student Affairs Work : Responding to Our Imperative. 
Washington, D.C.: National Association of Student Personnel Administrators, 1999). 

Given the breadth and depth of the impact on student learning of student interaction with faculty and staff outside 
of the classroom, the potential is great for faculty members working with students outside of the classroom to 
contribute to students’ individual growth and development. The potential is also great for faculty working with 
students to contribute to the success of their institution as a whole. In the words of Alexander Astin: 

Student-faculty relationship has a stronger relationship to student satisfaction with the college experience than 
any other involvement variable or, indeed, any other student or institutional characteristic. Students who interact 
frequently with faculty are more satisfied with all aspects of their institutional experience, including student 
friendships, variety of courses, intellectual environment, and even administratibn ofthe institution. Finding ways 
to encourage greater personal contact between faculty and students might increase students’ satisfaction with 
their college experience. (Astin, Alexander. Four Critical Years: Effects of College on Beliefs, Attitudes, and 
Knowledge. San Francisco: Jossey-Bass, 1978, 22) 

It is Astin’s view that purposeful, professional student-faculty relationships developed through the kinds of 
out-of-class activities that can be facilitated by student affairs staff members can contribute to student retention and 
academic success and to the fulfillment of an institution’s educational mission. Collaboration between faculty and 
staff in the development of educationally purposeful out-of-class activities can also contribute to the development 
of a culture of student involvement. George Kuh and John Schuh have composed a description ofthe characteristics 
of “an involving college” that depicts the role of faculty in promoting student involvement: 

An involving college has a clear, coherent mission and philosophy that communicate high but reasonable 
challenges forstudents buttressed by ethics of care and membership. Interpersonal distinctions are deliberately 
accentuated, or minimized, to attain the institution’s educational purposes and a clear, unwavering commitment 
has been made to become a multicultural campus community. The physical setting (rural, near a city, surrounded 
by a metropolitan area) is used to educational advantage by creating human-scale settings that discourage 
anonymity and provide numerous opportunities for participation in the life of the institution. A complicated web 
of cultural artifacts (history, myths, sagas, heroes/heroines, traditions, rites and rituals, subcultures, institution- 
specific language) underscores the importance of involvement and communicates to students “how the 
institution works.” Policies and practices hold students responsible for their own behavior and learning, blur the 
artificial boundaries between in-class and out-of-class learning opportunities, distribute resources consistent 
with the institution’s educational purposes, and enable subcommunities of students to flourish, such as ethnic 
or academic theme houses. In some subtle and not-so-subtle ways, faculty and staff promote student 
participation in educationally purposeful out-of-class learning activities. (Kuh, George D. and John H. Schuh, 
editors. The Role and Contribution of Student Affairs in Involving Colleges. Washington, DC: National Association 
of Student Personnel Administrators, 1991, 29) 

In the final analysis, the concept of collaborating for the common good is not really a new one in higher education. 
Despite the proliferation of specializations and specialists, there is already a significant degree of collaboration that 
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takes place in our institutions. One wonders, however, given the research on learning outside of the classroom, and 
the benefits to be gained by both students and their institutions, why more collaboration does not take place. With 
programs being implemented on our campuses such as learning communities, first-year experience programs, and 
service learning programs, we are perhaps at a critical juncture if we aim to make collaboration one of the hallmarks 
of higher education in the twenty-first century. If institutions are, in the words of Boyer, “In Search of Community,” 
and if we are truly concerned about “the common good," perhaps we are prepared to fully commit ourselves to the 
ideal of collaboration in the academy (Boyer, 1990). 

With this as the overall philosophical framework, this presentation will focus on the interpretation of collaboration 
from the perspectives of a president, a chief academic officer, and a chief student affairs officer. Specific strategies 
utilized to advance this approach, examples of collaborative efforts from three institutions, and perceived and realized 
benefits will be shared. 



Jack P. Ca/areso is President of Briar Cliff College in Sioux City, Iowa. 

Thomas V. Boeke is Executive Vice President and Academic Dean at Briar Cliff College in Sioux City, Iowa. 

Robert A. Bonfiglio is Vice President for Student and Campus Life at State University of New York College at Geneseo. 
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A New Path- 

Collaborating Outside Our Four Walls: 
The Good, the Risky, and the Oh-Oh! 



Jane Toot 



Introduction 

This is an era of decreasing resources and increasing expectations in two segments of our professional community, 
health care and higher education. When these two groups are involved with one another, such as in a teaching 
hospital, the needs can be exacerbated. However, the rewards can be significant when the involved parties forge 
creative alliances and exhibit a commitment to merge resources and jointly design programs. 



Description 

Such a commitment was undertaken in Grand Rapids approximately two years ago. In the fall of 1 999, two universities 
and three other organizations-Grand Valley State University [GVSU], including the School of Health Professions and 
Kirkhof School of Nursing; Michigan State University [MSU] College of Human Medicine; two health care systems, 
Spectrum Health and St Mary’s; and a center for graduate medical education-put their heads and their resources 
together to form a new entity, which they named the Grand Rapids Medical Education and Research Center for Health 
Professions [MERC]. The mission of this consortium is to enhance the health of the Grand Rapids community through 
the integration of high quality graduate medical education programs, medical student education programs, 
continuing education and continuing medical education programs, health professions education, and research 
relevant to the population of West Michigan (Springer, 2000]. 

Through the creation of this consortium, which addresses education and research on a region-wide basis, health care 
systems and universities are crossing boundaries in a noncompetitive manner. The first year of operation has been 
a challenging learning experience as members have charted a new path for integrated medical and health professions 
education and research. 



Activities (Current and Proposed) 

In the formation of MERC, several cultures, which previously operated under very different goals and objectives in 
regard to research and education, have been brought together. Each organization not only has focused upon its own 
agenda, but also has been successful by utilizing its own tried and true strategies. Not only does MERC need to bring 
these disparate groups to a shared vision, it must also develop environments that are appropriate and productive for 
each participant 

The general criteria for membership include [1] being a nonprofit organization; [2] having a part of the mission address 
medical education, health professions education, and/or research; and [3] being committed to improving the health 
of the residents of west Michigan. As the organization was structured, the board and all committees have 
representation from all of the full members. 



MERC is intent on the development of clinical research in west Michigan. One step toward this is the formation of 
a citywide institutional review board ORB]. Another anticipated support piece for clinical research, as well as 
education, is the development and improvement of library services. 

O 
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It is believed that MERC will foster new educational opportunities through the increase of educational links between 
and among campuses and health care centers. One such opportunity is the use of joint faculty clinical appointments. 
As has been indicated, MERC is already serving as a clearinghouse for scheduling clinical rotations for the medical 
school, some portions of the nursing program, and the physician assistant program. It is anticipated that this activity 
may expand to serve other health professions. 

MERC is focusing on faculty as well as student development. The College of Human Medicine at Michigan State 
University already uses problem-based learning (PBL). Such an approach is in the developmental stages in the School 
of Health Professions at GVSU. The faculty (both in-house and at the clinical sites) will require significant instruction. 
Furthermore, as MERC moves into interdisciplinary study with medical students, nurses, and other health professions, 
faculty development will receive considerable attention. 



Accomplishments 

MERC can claim numerous accomplishments in a short time span. Significant among these is [1) placing several 
students into clinical rotation in the disciplines of medicine, nursing, and physician assistant; [2) developing and 
implementing an efficient budget, which saved the hospital members ten percent; [3) increasing the quality and 
quantity of researchers for the research team; and (4) working on the establishment of a city-wide Institutional Review 
Board (IRB). 



Challenges 

While MERC has had an effective beginning, several challenges remain for this unique organization. Interdisciplinary 
and interinstitutional (health and educational) cultures involved with MERC must strive to develop integrated 
programs where appropriate. The citywide IRB has still to establish policies and procedures acceptable to all 
participants. It is clear that such a board will do much to expedite the approval of research projects; however, the 
diversity of rules and procedures in each institution make this a very complex proposal. Attention to disciplines beyond 
medicine has yet to occur as far as integrated research, continuing education, and education are concerned. Finally, 
the parameters of membership and the ramifications for involved organizational structures require clarification. 



The Problem 

As has been mentioned, during this founding period, MERC has accomplished several objectives regarding the 
educational and research environments for medical residents and students. In clinical education, more than 500 
students of medicine, nursing, and physician assistants have been placed. MERC has also made some changes in 
the procedures for the annual research presentation day so that allied health professionals and nursing students can 
participate. However, there have been no interdisciplinary efforts beyond this. The members do not view this as a lack 
of interest or intent on the part of MERC. Rather, it is considered a developmental stage for a work in progress. It is 
recognized that the shift from a paradigm of medical education to medical and health professional education will 
require significant degrees of flexibility and cooperation across disciplines and professions. 

Accordingly, for this organization to maintain its momentum, a formalized process utilizing interdisciplinary/ 
interinstitutional work teams should become standard operating procedure. Rather than just identifying projects, 
MERC needs to concurrently expend efforts on the process of working together toward reaching these objectives. 
Initially, this will be time consuming and difficult, but as stakeholders become familiar with the process, goals and 
objectives will be more efficiently reached and maintained. 



Step One: Communication 

A foundation of communication to support teamwork has been laid with the board of directors, the academic council, 
the continuing medical education committee, and the graduate medical education committee. While these groups 
have dealt primarily with issues surrounding education and research as they pertain to medical students and 
residents, their membership has brought an interdisciplinary viewpoint to the discussions. This interaction has helped 
foster communication and allows medicine and the health professions to gain knowledge about each other’s 
educational philosophies, strategies, goals, and objectives. 
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Step Two: Team Building 

A second step toward this interactive process of teaming is the commitment to the use of teams. There are two areas 
in which work teams would be useful. The first is to have the board of directors participate in the definition of 
membership parameters and the ramifications for member organizations. Such an activity would serve as an 
instrument for the development of a collaborative team effort. The principles that the board members would have to 
bring to the table include a belief in: 

o having a common purpose; 
o fostering the involvement of stakeholders; 
o developing joint outcomes from assessment; 
o coordinating activities of implementation and evaluation; 
o evaluating the effectiveness of their own team. 

The group must recognize that the process, rather than the final product reflects the success of a team. As this work 
evolves, the board will see how this process can be utilized to approach other issues and problems that come before 
the organization. 

The second area in which a work team could be advantageous is the academic council. This group is the logical body 
to propose and revieweducational and research directions for the organization. As in the case of the board, the council 
members must bring the same principles that foster collaborative team building to the discussions. 



Step Three: Conflict Management 

Finally, it is a truism that conflict is a fact of life. This is particularly true during organizational change. One author has 
likened the stages of organizational change to traveling through white water. Those of you who kayak or canoe can 
relate to this analogy. In this organization, where each member may have to lose some individual control in order to 
see long-term advantages, the likelihood of conflict is very real. The team facilitator and the member representatives 
must be very aware of the nature of conflict and know how to address it productively. Each member has probably been 
through conflict management, but providing succinct handouts would be helpful. 

Conclusion 

The potential of MERC is significant. It brings a unique dimension to the education and research activities for health 
care professionals in west Michigan. Realization of this potential is highly dependent on the ownership felt by the 
stakeholders. To create that sense of belonging requires each subgroup to do meaningful work. The team approach 
can provide the necessary environment for success as work groups address presented challenges. 

Is this type of collaboration good? Yes. Even in these early stages, we have seen better and increased communication, 
significant savings, more efficient clinical placements, and the very real potential for a citywide IRB. 

Is there a risk? Certainly, this is a possibility when several organizations combine to embark on a new structure in 
which trust and collaboration must be present as some control and perhaps some income are relinquished. 

The oh-ohs? To date, these have been to the effect of facing unforeseen opportunities rather than an overwhelming 
negative surprise. Currently, we believe our future is as bright as we care to make it. 
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Note 

A special "thank you” to James Springer, Ed.D., President and CEO of MERC, for his contributions to this presentation. 



Jane Toot is Director of the School of Health Professions at Grand Valley State University in Allendale, Michigan. 
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Accel It! 

Developing and Unleashing 
Creative Learning Potential 
In Your Classroom 



Vicki Osendorf 
Joan Barrett 



Workshop Summary 

Accelerated Learning is founded on the teaching methodology created by Dr. Georgi Lozanov, a medical doctor, 
psychiatrist, and educator. For many years, he was fascinated by how people learn and why some people learn faster 
than others. After many years of study, he concluded thatthe approach to learning was the single most important factor. 
He singled out very simple, yet, when applied to education and learning, very profound elements. Dr. Lozanov knew 
intuitively and experientially 40 years ago what is in fact the neurological and psychological way that we best learn. 

When asked to consider what makes a person intelligent, a common response may be a person’s ability to solve 
problems, utilize logic, and think critically. These typical traits of intelligence are sometimes lumped together under 
the label of "raw intelligence.” A person’s intelligence, traditionally speaking, is contained in his or her general 
intellect, how each comprehends, examines, and responds to outside stimuli, whether it’s solving a math problem or 
anticipating an opponent’s next move in the game of tennis. Our intelligence is our singular, collective ability to act 
and react in an ever-changing world. 

According to Lyelle Palmer, Professor of Education and Director of the Office of Accelerated Learning at Winona State 
University in Minnesota, the Accelerated Learning approach “adapts the presentation to the way people’s minds really 
work.” Dr. Palmer says that actual techniques and principles used in Accelerated Learning are not new, and many 
have been researched and used for years. He goes on to say that what is new is the total package and the sequence 
of presentation aided by the consistency of the teacher in preparation, presentation, and engagement of students.” 



Learning Cycle Optimized Learning 

o Creates a relaxed but attentive state of mind 
o Presents big picture overview 

o Presents material via three primary senses-visually, verbally, physically 
o Activates the learner to store to long-term memory 
o Demonstrates learning has occurred 
o Reviews and reflects on what has been learned 



Accelerated Learning is characterized by creating an environment that reinforces learning at both the conscious and 
q subconscious level. Carefully selected peripherals in the form of music, art, visuals, and objects contribute to the 
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overall learning environment. Care is taken to ensure conditions for optimal learning by creating a positive emotional 
climate that minimizes fear and threat and unleashes creative learning potential. Vivid memories are developed 
through fun, energetic, creative games, stories, imageries, and activities that significantly increase retention. Transfer 
of knowledge to long-term memory is facilitated by spending the majority of class time [approximately 70 percent) 
in application activities that stimulate the learning by immediate practice and feedback on new skills. 



Topics Addressed in the Session 

o Accelerated Learning as a teaching tool 

o Practical application of Accelerated Learning techniques with the participants 
o Training course for faculty— outcomes and plans to implement 
o Question/answers 



Presenter s Relationship to the Topic 

Joan Barrett, President of St. Cloud Technical College, and Vicki Osendorf, Program Manager for the Supervisory 
Management Program at the St. Cloud Technical College co-present. President Barrett shares her perspective on how 
Accelerated Learning benefits the entire campus. Vicki Osendorf, project manager and instructor for Accelerated 
Learning, shares outcomes of the two-credit course. 



The Need to Accel It! 

Many instructors are feeling pressured. Information is increasing exponentially. Higher educational expectations are 
coming from business and society, and budget restrictions are increasing. Demands are being placed on colleges 
to accomplish more in less time with quality results. The answer may lie in the program design and teaching methods 
that optimize the use of classroom contact time. 



Research Base 

Accelerated Learning is a systematic way of organizing and presenting instruction to stimulate different learning and 
processing styles. According to Howard Gardner [1999), Accelerated Learning has been described as "whole mind, 
whole body learning” that includes the following characteristics: 

o Engages both left-brain and right-brain activity 
o Uses the conscious and subconscious mind 

o Engages each of the senses, stimulating the body as well as the mind 
o Addresses different learning and processing styles 

Intelligence has been commonly regarded as general mental capacity comprised of logical and linguistic abilities. 
Some believe that intelligence is fixed at birth for life. In contrast, Howard Gardner’s [1999) definition of intelligence 
is "an ability to solve a problem, fashion a product, devise a process or provide a service that members of a culture 
would value, including problem finding.” Gardner believes that intelligence is a combination of fixed ability and 
developmental learning based on experience and reflections and can therefore be modified throughout life. The 
intelligences identified by Gardner [1983, 1999) include: 

1. Logical-mathematical-using reasoning and notational language efficiently 

2. Verbal-linguistic-expressing and communicating skillfully in speech or in writing 

3. Visual-spatial-visualizing and representing skillfully; maneuvering and traveling 

4. Body-kinesthetic-playing sports, conveying emotions, and using tools 
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5. Musical-rhythmic-playing a musical instrument, using musical notations, or composing music 

6. Intrapersonal-understanding oneself, one’s motivations and behaviors 

7. Interpersonal-using one’s sensitivity to other’s moods, motivations, and intentions 

8. Naturalist-ability to identify and classify patterns in nature 

Gail Heidenhain (2000), Delphin Leading by Learning consultant, defines Accelerated Learning as design technology; 
a way to facilitate that takes in the needs of students, and what the instructor needs to teach, and creates the best 
learning environment. She goes on to say that Accelerated Learning should be called enhanced learning as the 
learning comes faster at a slower pace, using all of the senses. 

The teacher’s role is to act as a guide, so that each participant can discover from within that person who has fun 
learning, who can relax, and who can use imaginative powers to learn, create, retain, interconnect, and do creative 
problem solving. The teacher sets up the environment and classroom that eliminates threat. Students become 
emotionally involved with learning and with other participants. We are now aware that emotions are critical to learning. 
Some neurologists believe that there are no memories without emotions. The instructor is a key element, not as a 
holder of all knowledge, but as a guide who gently and with great assurance leads the students to become 
empowered. Instructors become learner-centered, and the implications for learning are vast. 

Knowledgeable college and high school faculty and teachers of every level throughout North America, Europe, Asia, 
South America, Africa, and Australia now frequently use Accelerated Learning. Dr. Lozanov says fun, relaxation, 
imagination, and emotional involvement are the elements that create the most transformation and increase the ability 
to learn and retain. 

Traditional measures of intelligence in education assumed that our ability to learn and do things comes out of a 
uniform cognitive capacity. Student ability has been measured using IQ tests such as the Stanford-Binet test and 
Scholastic Aptitude Test (SAT). One might say that the dictates of American education today were based on student’s 
scores on a battery of intelligence tests taken from kindergarten through postsecondary. Advocates of traditional 
education continue to push this paradigm of Uniform Schooling, which is an educational system based on national 
standards and efficient, cost-effective assessment in the form of multiple-choice examinations with a number two 
pencil. 

Dr. Gardnerfirst introduced the theory of multiple intelligences in Frames of Mind (Gardner, 1983). He has been heavily 
involved in school reform efforts in the United States. In 1986 he began to teach at the Harvard Graduate School of 
Education and continues his work today. Gardner said in NEA Today (March 2000) that: 

Multiple intelligences (Ml) is a psychological theory about the mind. It’s a critique of the notion that there’s a 
single intelligence which we’re born with, which can’t be changed, and which psychologists can measure. It’s 
not based on test correlations, which most other intelligence theories are based on. 

The claim is that there are at least eight different human intelligences. Most intelligence tests look at language 
or logic or both-those are just two of the intelligences. The other six are musical, spatial, bodily/kinesthetic, 
interpersonal, intrapersonal and naturalist. 

I make two claims. The first claim is that all human beings have all of these intelligences. It’s part of our species 
definition. The second claim is that, both because of our genetics and our environment, no two people have 
exactly the same profile of intelligences, not even identical twins, because their experiences are different. 

This is where we shift from science to education. If we all have different kinds of minds, we have a choice. We 
can either ignore those differences and teach everybody the same stqff in the same way and assess everybody 
in the same way. Or we can say, look, people leam in different kinds of ways, and they have different intellectual 
strengths and weaknesses. Let’s take that into account in how we teach and how we assess. 

So, how should teachers who believe in your theory change their approach to teaching? Multiple Intelligence 
is a tool. It’s not a goal. That means that you have to decide what you want to teach, and that should be based 
on what you think is important. In my own work, I’m a proponent of teaching for understanding, which means 
going deeply into topics so that students can really make use of knowledge in new situations. This is very, very 
different from most teaching, where people memorize material andean reproduce it on demand, but can’t make 
use of it in new situations. That’s what understanding entails. If you favor education for understanding the way 
I do, then Ml can be extremely helpful. Because when you are teaching a topic, you can approach the topic in 
many ways, thereby activating different intelligences. 
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You can see that I'm very much in opposition to the current state and national trends, which create more tests, 
often of a short-answer sort, favoring coverage or non-coverage and not probing deeply into what people really 
understand. . . . Frankly I don’t care what intelligence or intelligences people have. I care whether they can do 
things which we value in our culture. ... I think our assessments ought to focus on the kinds of things we want 
people to understand, and they ought to give people a chance to perform their understandings. . . . Assessment 
is fine. Even standardized assessment is fine, if it looks at things which are important and allows us to probe in- 
depth what people understand. 



Implementing Accel 

In 2000, the Supervisory Management Advisory Board at the St. Cloud Technical College assisted the faculty in 
reviewing courses, assisted in determining which courses would be merged, and identified new names for these 
courses. They reviewed and assisted in modifying course syllabi. Their input was invaluable as they shared their 
perspective from a business standpoint. 

In February, March, and April 2001, instructors from various disciplines throughout the campus who teach the 
Supervisory Management program are participating in a two-credit training program for Accelerated Learning 
[Accel It!). Upon successful completion, they will be certified to teach in the accelerated format. 

Following the training, instructors will work in teams to convert curriculum from the standard formatto an accelerated 
one. The Supervisory Management program will formally convert to an Accelerated Learning format in fall 2001. 

The college seeks to become a training resource center for Accelerated Learning for the State of Minnesota. 
Interested faculty and staff would meet or communicate on an ongoing basis to read related publications and discuss 
ideas. In the process, they would continue to examine their own intelligences and how that effects their style of 
teaching. We want to create a clearinghouse where instructors can share and discover new ideas from others. We 
believe this network would support instructors in implementing the newly learned concepts. 

If there are students you are not reaching, or if you struggle with getting your topic across clearly, give some 
Accelerated Learning concepts a try and see if they make a difference. Gardner said in his interview with NEA: “Indeed 
I often say that if anybody doesn’t believe in multiple intelligences they should go on an automobile ride with three 
different people and get totally lost and see how each person tries to get back home. Often that experience will make 
them an instant convert to the theory of multiple intelligences. People do not think the same way." 



Case Study on Accelerated Learning 

Marla Amborn, currently at Century College in White Bear Lake, Minnesota, applied Accelerated Learning concepts 
to Supervisory Management Programs in the Wisconsin postsecondary system. Students participated in a cohort and 
reported benefitting from: 

o teaching methodologies that built self-esteem and critical thinking skills while optimizing learning styles and 
skills; 

o a collaborative student experience that built team skills readily transferable to the workplace; 

o employer involvement and ownership in helping participants to successfully develop skills needed to advance 
in their career; 

o being able to earn an associate degree on a part-time basis in less time than the traditional approach 
[two-and-a-half rather than six to seven years); 

o a pre-determined schedule that students can depend on with no canceled classes, a consistent class meeting 
time, and a graduation date identified before they enroll in their first class. 



The College benefited from: 




o increased graduation rates for Supervisory Management students; 

o improved teaching skills of instructors who can use accelerated methodology in other program areas and 
business and industry training; 
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o less staff time lost by setting up classes, promoting them, ordering books, etc., and then having to cancel the 
class; 

o more productive working relationships with local employers; 

o public perception as an innovative leader in education responsive to the changing needs of employers; 

° the ability to compete with other postsecondary institutions in the area that offer similar programs. 

Employers in the community benefited from: 

o developing fully-trained supervisors in less time; 

o working closely with their employees to develop the specific skills needed in their workplaces; 
o class assignments used to accomplish projects needed by the company; 

o a greater pool of fully trained supervisors who are graduates of the Supervisory Management Program in the 
local workforce. 



Further Case Studies 

With the implementation of Accelerated Learning, graduation rates for the Supervisory Management Program at 
Lakeshore Technical College in Wisconsin increased from 3-4 to 30-36 graduates per year. This increase in the 
graduation rate accounted for 90 percent of the students enrolled in the program. Accelerated Learning proved to 
be one of the best methods of delivery for adult students because it can cut the student contact time in half. The 
Wisconsin Technical College Board has approved guidelines for Accelerated Learning that require the same course 
competencies be achieved as in the standard length program. 

Mindy Kornhaber at Project Zero has been investigating 42 different schools that have been using Ml theory for at 
least three years. Those schools report a lot of success with students on both hard measures-how they do on tests- 
and on softer measures, like absenteeism. 

David Meier and Mary Jean Gill did a study at Bell Atlantic in which they converted standard courses to an accelerated 
format. Training time was cut in half, while job performance and student satisfaction increased substantially. 

Charles Connley at Iowa State University routinely dismisses his class at midterm, having covered more materials than 
other professors who teach the same course. His students regularly perform at equal or greater levels on final exams. 

There is a training workshop called The Accounting Game in which participants use the transactions of a lemonade 
stand to learn the principles of accounting. In a one-day workshop, they cover all of the principles typically taught 
in an introductory college accounting course. 

Dramatic results have been found when applying Accelerated Learning methods to the study of foreign language, 
reducing the time it takes to learn a foreign language from years to months. 



Conclusions 

Based on research and demonstrated results, we believe that Accelerated Learning is a teaching methodology that 
is well worth pursuing. Our hope is to first reach those students who struggle and have a difficult time understanding, 
and to enable all students to fully involve themselves in the learning process and experience a new joy in learning. 
St. Cloud Technical College seeks to create a learning community that values and celebrates diversity. 



References 

Campbell, L, and B. Campbell. 1996, 1999. Teaching and learning through the multiple intelligences. Needham Heights, MA: 
Allyn & Bacon. 

Gardner, H. 1983. Frames of the mind: The theory of multiple intelligences. New York: Basic Books. 



Chapter 2. New Designs in Higher Education. New Designs to Improve Student Learning / 25 



Gardner, H. 1999. Intelligence reframed. New York: Basic Books. 

Heidenhain, G. 2000. Accelerated learning in organizations. Georgia: Delphin, Leading by Learning. 

Jensen, E. 1996. Brain based learning. California: Turning Point Publishing. 

Kagan Cooperative Learning. 1997. Multiple intelligences smart card. 

Meier, D. 2000. The accelerated learning handbook. New York: McGraw-Hill. 

Moller, L. 1991. What is this thing called accelerative learning? Colorado: Boulder Center for Accelerated Learning. 
Weiss, S. 2000. NEA Today Online, March, <http://www.nea.org/neatoday/9903/gardner.html>. 



Vicki Osendorfis Program Manager, Supervisory Management, at St Cloud Technical College in St Cloud, Minnesota. 
Joan Barrett is President of St Cloud Technical College in St Cloud, Minnesota. 



0 

ERIC 



35 



26/ A Collection of Papers on Self-Study and Institutional Improvement 2001 



Appendix 



Accelerated Learning 
Resources 



Dr. Howard Gardner 

Harvard Graduate School of Education 
201 Larsen Hall 
14 Appian Way 
Cambridge, MA 02138 

Leading expert on the theory of Multiple Intelligences 



Winona State University 

Dr. Lyelle Palmer 
Department of Special Education 
Office of Accelerated Learning 
Winona, MN 55987 
507-457-5366 

Conducts undergraduate and graduate courses in 
Accelerated Learning and sponsors annual conference. 



St Cloud Technical College 

Vicki Osendorf, Supervisory Management 

Dr. Joan Barrett, President 

1540 Northway Drive 

St. Cloud, MN 56303-1240 

320-654-5000 

vao@cloud.tec.mn.us 

NCA workshop presenters. Currently converting 
Supervisory Management Program to Accelerated 
Learning. 



Center for Accelerated Learning 

Dave Meier 
1 1 03 Wisconsin Street 
Lake Geneva, Wl 53147 
262-248-7070 
www.alcenter.com 

Trainer and consultant on Accelerated teaching methods. 



Century College 

Marla Ambom 

Continuing Education and Customized Training 
3300 Century Avenue North 
White Bear Lake, MN 55110 
651-779-3940 

Trainer and consultant on Accelerated teaching methods. 



Lakeshore Technical College 

Robin Butler, Supervisory Management 
George Grinde, Dean of Business and Marketing 
1290 North Avenue 
Cleveland, Wl 553015 
414-458-4183 

Developed first Accelerated Learning Supervisory 
Management Program in Wisconsin Technical College 
system. 

International Alliance for Learning 

Dr. Nancy Omaha Boy, President 
Box 26175 

Colorado Springs, CO 80936 

800-426-2989 

www.ialearn.org 

International Organization specializing in 
Accelerated Learning. 



Journal for the Society of Accelerated Learning 

Rutgers University 
406 Penn Street 
Camden, NJ 08102 

http://camden-www.rutgers.edu/Camden/TEC/JALT/ 

JALT.html 

Journal connected to the International Alliance for Learn- 
ing and Teaching 



Creative Training Techniques International 

The Bob Pike Group 
7620 West 78 th Street 
Edina, MN 55439 
800-383-9210 
Bobpikectt.aol.com 

Trains in Accelerated Learning methods and provides 
various resources such as books and newsletters. 
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Lessons Learned from 
General Education Revision 



William D. Jenkins 
Janice G. Elias 



The process of general education reform began at Youngstown State University with a North Central visiting team 
report that identified YSU’s distribution model as a concern. In the fall of 1 993, a committee was appointed to examine 
the development of a goal-driven model. Seven long years later, in the fall of 2000, YSU successfully implemented 
a new general education model. The new model (shown in the accompanying chart) is a radical change from the 
department-based distribution model previously followed. It is goal-directed rather than based in the disciplines. It 
extends across four years, synthesizes more than one goal in each course, integrates general education into major 
courses, and provides a relatively similar experience for each student. This paper shares some of the lessons learned 
from this complex undertaking. 



Ownership of General Education 

In the fall of 1993, the Dean of Arts and Sciences appointed a college-wide committee to develop goals for general 
education. Although the Academic Senate approved the goals in the spring of 1994, there was much concern about 
representation on theTask Force that would continue the work. Traditionally, liberal arts and sciences have provided 
most of the general education. Professional schools, such as Education and Engineering, provided career-related 
education. Should decisions about general education be the province of those usually responsible for delivery? Or, 
because the general education program affected all students and programs, was it a campus-wide responsibility? 
Is the assumption that professional schools can’t contribute to general education well-founded? Who owns general 
education in a comprehensive university? 

Of particular concern to the professional schools were the pressures from accrediting agencies, employers, and 
licensure exams for specialized knowledge. The need to squeeze in content made these schools wary of any increase 
in general education credits. They also wanted supporting course work in the arts and sciences to do double duty 
for general education and the major. Since accrediting agencies are seldom as prescriptive as some campus programs 
claim, it is helpful to look at the organization’s accreditation criteria. Nevertheless, faculty from professional schools 
have legitimate concerns about the balance between general and professional curricula. The provost and the 
Academic Senate recognized those interests by appointing a Task Force on General Education with representatives 
from all colleges. The result has been greater integration of general education with the major in four ways. 

0 “Intensive” courses. One component of the model, Essential Skills, requires students to take introductory 
writing, oral communication, and mathematics courses. Although basic courses provide a foundation, it is the 
recurrent use of the skills that enables a student to acquire proficiency. Because YSU faculty agree that 
students should develop writing, speaking, and critical thinking skills throughout the entire curriculum, one 
means of bringing the professional colleges into general education was the addition of “intensive" courses 
that engaged the student in skill-related assignments for 30 percent of the grade. These courses may be in 
the major and can be proposed by any department. Integrating these goals into major courses strengthened 
general education without drastically increasing the required credits. 

Although faculty generally agree with the principles above, the application of these principles often conflicts 
with other beliefs held by the faculty that content coverage is more important than skill development, and oral 
presentations take too much time. The most difficult to fit in, according to faculty, is the oral presentation. 
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Hence, we have begun to consider whether the assignments might be spread over more than two courses 
within the major. For some departments the task is simple; they may already include these skills. For others 
it requires an adjustment considered too severe by some faculty members. 

0 Capstone course. Students must also take a capstone course in the major that requires them to demonstrate 
mastery of the essential skills. With the combination of general education, essential skills courses, intensive 
courses in the major, and the capstone course, students are engaged in significant writing, speaking, and 
critical thinking activities throughout the undergraduate experience. 

0 Substitute courses. As the model developed, an additional dilemma arose. Courses in mathematics and the 
natural sciences designed to serve a general student population were not appropriate for math, science, 
engineering, and health professions students. Their professional curricula already required a significant 
amount of math and science. An approval process was developed to review and substitute more advanced 
courses that also addressed the general education goals. 

0 General education courses offered by professional schools. One of the most significant changes was 
acceptance of the principle that any department could propose courses. Some liberal arts departments saw 
this development as a threat, while professional programs saw an opportunity. Faculty from liberal arts 
questioned whether professional school faculty were too technically oriented to provide general education. 
The solution was a proviso that all course proposals demonstrate that the course satisfied specific goals of 
general education, was appropriate for the general student body, and was accessible to non-majors. In 
addition, the Academic Senate approved criteria that provided a clearer definition of the goals. Using these 
guidelines, some professional departments have been successful in having courses approved for general 
education, and others have not For example, the Department of Human Ecology course in nutrition was 
approved in the category of Personal and Social Responsibility. A course in engineering economics was not 
approved because it was too narrow and designed primarily for engineering students. 



The Politics of Curricular Reform 

There are many sad tales told about academic senates. Numerous committees have brought policy recommendations 
to those bodies, only to have them rejected. Committee members, who believe that they have studied the issues and 
made sound judgments, become angry and disillusioned when their senate colleagues do not endorse their work. Our 
Task Force recognized that success in the Senate would depend primarily on two factors: CO the power of expertise, 
and [2] skill in consensus building. The first was easier than the second. The provost provided funds for attendance 
at national conventions, buying books, and bringing experts to campus. We were prepared to explain how and why 
such reform should occur. 

Managing the political process was more complicated. The Task Force knew that its broad representation was 
necessary, butnot sufficient, to build the consensus necessary for radical change. The members, therefore, conducted 
open forums, met with each academic department as well as deans’ advisory councils, gave periodic reports at the 
Academic Senate, solicited written comments, and invited many individuals to attend Task Force meetings to discuss 
their concerns. In addition, the TaskForce chair kept students informed through presentations at Student Government 
meetings. Another component of our strategy, as recommended byJerryGaffof AACU, was the presentation of reform 
in pieces rather than as a total package. We recommended the program to the Senate for confirmation in stages: the 
goals, whether to adopt a core of courses, the distribution of courses within domains, and the criteria for certification 
of courses within the domains. 

A central principle followed was that no policy should go before the Senate before it had been fully vetted throughout 
the university. There were debates, and some changes, on the floor of the Senate, but all the key components of the 
new model passed. There is no doubt that this approach was time consuming and exhausting, but YSU did not have 
the degree of conflict experienced at some universities, nor was a restart required because the Academic Senate 
defeated the proposals. Taking an adequate amount of time for self-education and consensus building was essential. 
The new model passed the Academic Senate in the spring of 1998 and was implemented in fall 2000. 

□ Core Curriculum? 

The path to a consensus was not always smooth. While there may be agreement in principle to the ideal of a goal- 
driven program, the "devil is in the details,” and disputes did arise regarding core requirements and college 
autonomy. It is very difficult for a university to reach agreement on a core. The number of faculty with vested 
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interests in existing courses and their strong discipline orientation make itdifficult for them to agree that any single 
set of core courses best addresses the goals or to join together to develop interdisciplinary courses. When a single 
department attempts to secure a required course, others may question whether they are motivated by concern for 
the well-being of students or a desire to guarantee student credit hours and ultimately resources to the department. 
For example, the Political Science Department was concerned about the declining participation in elections. They 
proposed that a course on citizenship be required of all students. In addition to resource questions tthe disparity 
between the number of sections needed and the number of faculty in the Political Science Department), many 
faculty felt that equally good arguments could be made for requiring other courses. The Political Science 
Department was unable to gain support for their proposal. Consensus developed that specific core courses would 
focus on the essential skills of writing, oral communication, critical thinking, and mathematical reasoning. The 
remaining goals would be addressed by student choices from a restricted number of courses organized into four 
domains. 

□ College Autonomy 

Some individuals concerned about breadth suggested that students should not be allowed to “double dip," to count 
the same course toward fulfilling the major and general education requirements. For the same reason, others 
objected to department requirements that their students take specific general education courses. The Task Force, 
sensitive to issues of departmental autonomy, took the position that major requirements were the purview of the 
colleges and departments so long as all curricula included the required general education model. The College of 
Arts and Sciences voted not to permit students to count courses in their major toward meeting the general 
education requirements. The professional schools permit the same course to be used for both. 



Resource Constraints 

Faculty are idealists when it comes to what they teach. Often they fail to consider the relationship between curriculum 
and the budget. Thus, when general education reform occurs, a conflict ensues because only so many courses can 
fit into the model. 

The Task Force consulted with the administration early in the process and determined that the university could not 
support a program that required more resources than the current one. That set the stage for two principles that were 
generally followed: (1) the portion of the curriculum devoted to general education could not be drastically increased, 
and [2) the individual components of the program could not require major shifts in resources. An early understanding 
of the resource constraints was essential to developing a practical model. 

For example, one clash involved the computer science department, which believed that the goal “acquire, process, 
and present quantitative and qualitative information using the most appropriate technologies, including computers” 
meant that all students should learn the underlying principles of computer operation. They did not feel that the 
incorporation of word processing and use of the Internet in the introductory writing courses was adequate to meet 
the goal. In this case, the department involved was not able to convince the rest of the faculty that its ideal should 
be theirs, particularly since the department did not have the faculty resources to staff such a requirement. 

On the other hand, there appeared to be widespread support for a required speech course. The Communications 
Department was enthusiastic, but the General Education Task Force was reluctant to increase the hours in general 
education and to face the problem of resources. The Communications Department appealed to the idealism of the 
senators and argued that the cost would be small. Other departments asked where the money would come from. 
Ultimately the Senate endorsed this addition to the model, realizing that some departments might lose faculty 
positions and budget to the Communications Department, which would need more resources. 

The limitations posed by resources stimulated creative solutions and resulted in a model that is more integrative than 
it might have been otherwise. For example, with 13 goals and a ceiling on the number of courses, it was clear that 
every general education course would have to address more than one goal, and the goals would need to be 
incorporated into major courses as well. The resource constraints reinforced the development of a coherent program. 



Program Administration and Governance 

The design of the administrative structure and governance flows from the early decision that general education is not 
a program that belongs to any one college, but that it belongs to the entire university. To secure passage of the model, 
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it was necessary to define the governing structure. A General Education Committee that approves courses and 
recommends policies was created. The committee includes a representative elected from each of the six under- 
graduate colleges, and one appointed representative from each of the five areas of general education. This 
representation constitutes a compromise between the professional schools and the usual providers of general 
education courses. The committee is chaired by the Coordinator of General Education who performs administrative 
duties and reports directly to the Provost. Although a dean of general education or general studies might have more 
power and funding, the Task Force believed that leadership should come from a faculty member. The administration 
has provided a modest budget, an office, and secretarial help for the Coordinator. 



Recommendations 

After seven years, YSU has made significant changes in general education. That process has been frustrating and 
difficult, yet challenging and stimulating. From this experience, we have culled the following suggestions for those 
undertaking general education reform: 

1 . Take adequate time for a self-education stage for a significant number of people on campus. 

2. Be realistic about the length of time it will take to develop consensus on a new program. 

3. Include university-wide representation on the committee that develops a new program. 

4. Take time to understand the concerns expressed by the professional schools and disciplines and seek to find 
mutually acceptable solutions. 

5. Develop the program in steps, seeking consensus first on the goals and philosophy as a basis for future 
decisions. 

6. Clarify the resource issues to avoid developing a model that cannot be supported. 

7. Develop an organizational structure that promotes university-wide ownership and governance of the program. 

8. Determine the goals of the general education program before designing the model or choosing specific 
courses. 



William D. Jenkins is Coordinator of General Education at Youngstown State University in Youngstown, Ohio . 
Janice G. Elias is Assistant Provost for Planning at Youngstown State University in Youngstown, Ohio. 
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Appendix 

Youngstown State University General Education Model 



Goals 


Requirements 


1. Write and speak effectively 

2. Acquire, process, and present quantitative and 
qualitative information using the most appropriate 
technologies, including computers 

3. Reason critically, both individually and 
collaboratively, draw sound conclusions from in- 
formation, ideas, and interpretations gathered from 


A. Essential Skills 

2 introductory writing courses 
1 oral communication course 
1 introductory mathematics course (requirement may 
be satisfied by a higher-level mathematics course or 
score on the placement exam) 


various sources and disciplines; and apply those 
conclusions to one’s life and society 

5. Comprehend mathematical concepts and reason 
mathematically in both abstract and applied contexts 


2 upper division writing-intensive courses, preferably 
in the major or minor 
2 oral communication-intensive courses 
2 critical thinking-intensive courses 




1 upper division approved capstone course, usually in 
the major 


6. Understand the scientific method; forming and 
testing hypotheses, as well as evaluating results 

7. Realize the evolving relationships among science, 
technology, and society 

1 3. Understand and appreciate the natural environment 
and the processes that shape it 


B. Natural Science 

2-3 courses* from a list of courses that meet goal 13 
and in addition goal 6 or 7; one must have a lab 
component 


8. Grasp and appreciate artistic expression in multiple 
forms and contexts 


C. Artistic and Literary Perspectives 

2-3 courses* from a list of courses that meet goal 8, 
and one additional goal 


10. Understand the development of cultures and 
organization of human societies throughout the 
world and their changing relationships with 
Western society 

11. Evaluate the impact of theories, events, and institu- 
tions on the social, economic, legal, and political 
aspects of society 


D. Societies and Institutions 

2-3 courses* from a list of courses that meet a combina- 
tion of two goals from 10, 11, 12 


12. Comprehend and appreciate the development of 
diversity in America in all its forms 


* Number of courses from B, C f D must total 8 


4. Understand the personal and social importance of 
ethical reflection and moral reasoning 

9. Understand the relationships between physical, 
mental, and emotional well-being and the quality of 
life of the individual, the family, and the community 


E. Personal and Social Responsibility 

2 courses from a list of courses that meet either goal 
4 or goal 9 in combination with one other goal from 3-1 3 




F. Special Topics and Electives 

1 course that combines goals in ways not listed above, 
or 1 course from mathematics, B, C, D, or E above 
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Common Courses and Capstones: 
A New General Education Initiative 
at William Jewell College 



Randall Morris 
Judith Dilts 

This paper intends to introduce readers to the new general education initiative at William Jewell College. To that end 
we propose to provide a brief account of the transition we have made from a typical distribution model toward a 
sequential core, one that includes several common courses taken by students in their freshman and senior years. We 
will include enough information to familiarize program participants with the structure and objectives of the new 
program and conclude by identifying seminal issues to be addressed at the meeting in Chicago. 

Until fall 1996, William Jewell offered students two options for satisfying their general education requirements. The 
large majority of students pursued a traditional distribution track. The courses satisfying general education were 
invariably discipline specific courses that were often created to serve the needs of the majors. This led us to reflect 
on our goals and objectives for general education and to ask ourselves whether this approach was enabling students 
to achieve them. Very often, the courses were intended to introduce students to the basic ideas of a discipline, 
providing thereby a foundation to be built upon by later courses in the sequence. While useful for students who would 
become majors, the educational needs and interests of the non-majors taking the course for general education credit 
were not immediately considered. Furthermore, each course was entirely self-contained, designed without consid- 
ering what other general education courses students were taking. By the very nature of the distributional approach, 
this was inevitable. There was no way of knowing what courses students in your classes had taken or were likely to 
take in the future; consequently, it was impossible to construct a course that related directly to any other course in 
the curriculum. No effort was made, or even could be made, on the part of students or faculty to integrate the content 
of such discrete courses. Yet we claimed to be preparing students to enter a complex world in which issues are best 
addressed from a multitude of perspectives. 

The distributional general education curriculum was not the only one available for students. A small number of 
students opted for a sequential program called Foundations forthe Future. The Foundations program offered students 
a strict sequence of six courses that were team taught and interdisciplinary. The general themes of the program were 
values and decision-making, and each course had such a focus. Students chose to enroll in the Foundations program 
as incoming first-year students, and took the courses through to a capstone course their senior year. While the 
program was successful and offered to those planning our new general education program an excellent model (e.g., 
a program covering a student’s four years, a capstone course, experience with interdisciplinarity], it was not adopted 
as the general education program for the college in large part because each course was team-taught, and we didn’t 
have the faculty to staff such a labor-intensive program. 

Finally, the basic problem that led to our revising the general education program at Jewell was assessment. How could 
we assess student achievement underthe currentdistributional system in orderto know whether ourgoals were being 
achieved? The assessment subcommittee of the Curriculum and Educational Policy Committee concluded that “the 
general education at William Jewell did not constitute a coherent and assessable program.” They mandated the 
college to construct a new program for general education. 

The process of revising general education began in 1994 and was implemented in 1996. The result is a thirty-eight 
credit hour program called The Responsible Self. The program consists of three tiers. The introductory level begins 
with a collection of courses taken by all first year students. In addition to the typical courses in communication and 
mathematics, all first-semester students take a seminar called The Responsible Self. The second level offers students 
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a choice of interdisciplinary courses grouped around four topic areas: culture and traditions: science, technology, and 
the human experience: power and justice: and sacred and secular. Finally, the capstone course is intended to build 
bridges between the program’s previous levels and the student’s study of an academic major by applying questions 
of responsible, ethical citizenship to critical problems facing society. (The table at the end of this paper provides an 
overview of the general education program.) In addition to these courses, all students are required to complete two 
credit hours of physical education activities. B.A. students must also complete one intermediate foreign language 
course, and B.S. students must take an additional course at level II with a multicultural emphasis. 

One benefit of the revised program is that it provides an opportunity to expose students to greater levels of complexity 
in relationship to themes and ideas with which they are already familiar. Our goal in the program is to introduce 
students to ideas and/or texts that they revisit throughout the program. In our presentation we will offer examples 
of this connectivity in two important areas, science and ethics. 

Secondly, because students in later courses revisit ideas and texts introduced at an earlier level, we have additional 
means to measure student achievement. In addition to assessing student writing, we hope to be able to evaluate their 
critical thinking skills as demonstrated by their ability to deal with related material at a deeper level. 

At the same time, despite the potential of the new system, we have been forced to recognize that faculty development 
issues threaten the successful implementation of this program. Whereas in our distributional model, faculty were able 
to use courses designed to satisfy the needs of their majors, they are now expected to develop interdisciplinary 
courses that meet specific general education goals and objectives. This asks a lot of faculty. We have discovered that 
not all faculty members have the ability, much less the desire, to learn and teach in an interdisciplinary program. And 
even if the institution has enough capable faculty members, the administration needs to commit significant resources 
to initial and ongoing faculty development. Finally, it is tautologically true that if you intend to use content matter to 
connect elements of a general education program, faculty members cannot design courses in isolation. This was one 
of the problems of our distributional approach. Faculty teaching in the new program need to carefully and deliberately 
link their courses to those that come earlier in order to build on the common experience of students. This has proven 
to be problematic. It impinges on faculty autonomy. There is a need for strong leadership, both from faculty and 
administration, to hold faculty members accountable to the goals and objectives of general education. 



Randall Morris is Professor of Philosophy and Coordinator of the Responsible Self (Gen 100) Course at William Jewell 
College in Liberty ; Missouri 

Judith Dilts is Dr. Burnell Landers Chair of Biology and Professor at William Jewell College in Liberty ; Missouri 
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Appendix. William Jewell College General Education Program 



Level 1 

Students are required to complete each of the following courses before progressing to Level II. 


GEN 100 

The Responsible Self 
C4 credit hours) 


GEN 101 

Ora! Communication 
[4 credit hours) 


GEN 102 

Written Communication 
[4 credit hours) 


GEN 103 

Math Model Building 
and Statistics 

Or 

GEN 104 

Statistics and 
Applied Calculus 
[4 credit hours) 


Level II 

Students are required to take one course from each of the following categories 
except for the category to which their academic major is most closely related. 


Culture and Traditions 

[4 credit hours) 

GEN 200 

United States Pluralism 

GEN 201 

Divas, Death and Dementia 
on the Operatic Stage 

GEN 202 

Performance Studies 

GEN 203 

History and Philosophy 
of Science 

GEN 204 

Cultural Values and 
Visual Art 

GEN 205 

La Doulce France 

GEN 206 

The Chicano Experience 

GEN 207 

The WJC Fine Arts 
Program Events 

GEN 208 

Women Writers of 
World Literature 

GEN 209 

Cultures in Transition 

GEN 210 

Film Worlds 

GEN 211 

Seeing is Believing: 

The Iconography of 
Suffering and Compassion 


Power and Justice 
in Society 

C4 credit hours) 

GEN 275 

Social Problems 

GEN 276 

Human Development: 
Psychoanalysis and Literature 

GEN 277 

Deviance and Discipline: 
Crime and Punishment in 
Historical Perspectives 

GEN 278 

The Worlds of Islam 

GEN 279 

Economic Development 
and Cultural Change 

GEN 280 

Hitler's Germany: 
Problems of Power and Justice 

GEN 281 

Medicine, Money, and Morals 

GEN 282 

Constitutional Questions 

GEN 283 

Synopsis of United States 
History and Government 


Sacred and Secular 

C4 credit hours) 

GEN 225 

Law, Gospel, and Moral 
Philosophy in the Traditions 
of the Reformation 

GEN 226 

Religion and Meaning 

GEN 227 

Religion, Meaning, 
and Reality 

GEN 228 

Religion as Literature 

GEN 229 

Christianity and Tyranny 

GEN 230 

Religion and the Holocaust 

GEN 231 

Biblical Messianism and 
Handel's Messiah 

GEN 232 

Relationships: Psychological, 
Religious, and Societal 
Perspectives 


Science, Technology, 
and the Human 
Experience 

[4 credit hours) 

GEN 250 

Earth beat 

GEN 251 

The Science of Fibers 

GEN 252 

DNA: Politics, Law and Ethics 

GEN 253 

Energy: Its Sources and 
Responsible Use 

GEN 254 

The Mind: 

The Master Pharmacist 

GEN 255 

Sports Science: Physics 
Applications and Ethical Issues 


Level III 

Students are required to take one of the following capstone courses during their senior year, 
dependent upon completion of Level II. 


GEN 401 

Birth by Any Means? The Ethics 
of Reproductive Technology 


GEN 402 

Capitalism, Socialism, 
and Democracy 


GEN 403 

Plague, Piety, and Public Policy 


GEN 404 

State and Society 
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Chicago ENLACE: 
Building Communication with 
Stakeholders in the 
Latino Students' Education K-16 



Joaquin Villegas 
Santos Rivera 
Estela Lopez 



The College Board’s 1 999 Report of the National Task Force on Minority High Achievement defines its primary mission 
as “to develop recommendations for how several segments of American society, ranging from senior higher education 
officials to minority parents, can work more effectively to increase the number of underrepresented minority students 
who can achieve at very high levels academically.” The Task Force Report, entitled Reaching the Top, indicates that 
“in only a few decades, our society has developed a number of promising strategies and strong leads for improving 
educational outcomes for underrepresented minorities.. .. Nevertheless, much too little attention has been given by 
our society to the limited presence of African Americans, Latinos, and Native-Americans among the nation's most 
academically successful students. As a result, relatively few educational strategies have been demonstrated to help 
solve this issue” (College Board, 1999, Reaching the Top: A Report of the National Task Force on Minority High 
Achievement New York: College Board). 

In consonance with the findings of this report, the W. K. Kellogg Foundation in 1 999 launched an educational initiative 
under the name ENLACE: ENgaging LAtino Communities for Education. This initiative was aimed to increase 
opportunities for Latino children and youth to prepare for, pursue, and succeed in postsecondary education. Through 
ENLACE, Kellogg will strengthen efforts focused on Latino access to and success in college by: 

o strengthening selected Hispanic-serving institutions, public schools, and community-based organizations to 
serve as catalysts and models for systematic change in education; 

o supporting higher education/community partnerships that increase community involvement and educational 
success among Latino students; 

o supporting the creation and/or implementation of educational models based on best practices that improve 
enrollment, academic performance, and graduation of Latino high school and college students; 

o facilitating the expansion and sustainability of successful programs through strategic planning, networking, 

leadership development and policy efforts; and 

o disseminating information about and publicizing successful models to key stakeholders to stimulate changes 
in policies and practices related to the education of Latinos. 

In order to address the ENLACE initiative and achieve its five objectives, the W. K. Kellogg Foundation delineated three 
stages or phases named: (1 ) identifying and developing partnerships; (2) implementation; and (3) institutionalization, 
sustainability, and dissemination to be pursued by institutions of higher learning. 

Northeastern Illinois University (NEIU) viewed itself in a unique position to lead Kellogg's ENLACE initiative in the 
Midwest of the United States. As a state-supported commuter institution and the only four-year public Hispanic- 
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serving institution (HSI) in Illinois, NEIU serves a culturally diverse student body of approximately 10,035, of whom 
26.6 percent are Latinos. According to a Black Issues in Higher Education special report entitled “The Top Degree 
Producers” published in July 1998, NEIU ranks sixteenth in the top grantors of education degrees to Latinos. 

In fall 1999, NEIU competed with 125 HSI members from the Southwest, Midwest, California, Florida, New York, 
and Texas. NEIU was awarded a Phase I initiative with a one-year planning grant in order to "formulate creative 
approaches and models to prevent high school dropout and support Latino/a youth to enter and complete college.” 
This effort required a partnership with public schools and community-based organizations in order to include these 
community voices in the design plan to identify the strengths and assets that each partner can bring into the 
ENLACE initiative. 

In spring 2000, NEIU became the catalyst force to convene groups of teachers, parents, students, and community- 
based business and professional people representing the full range of Latino cultures present in Chicago-Mexican, 
Puerto Rican, Cuban, and Central and South American-to address the barriers that prevent Latino students from 
achieving educational parity with the rest of American society. The Chicago ENLACE Partnership, as it will be named 
by the partners, targeted three critical issues that need to be addressed to improve the educational pipeline and 
increase opportunities for Latinos to enter and complete college: 0J the need to expand the existing college/ 
community partnerships that address high school dropout to support the education of Latino youth through college; 

[2] the need to perceive the language and culture of Latinos as an asset in educational endeavors, not a barrier; and 

[3] the need to develop a concerted effort to involve not only individual students but also family groups in Latino 
students’ goal setting and planning for postsecondary education and careers. 

The Chicago ENLACE Partnership’s operating structure was built on existing collaborative efforts that NEIU has had 
developed through its Chicago Teacher Training Center. Two clusters were organized in predominantly Latino 
Chicago neighborhoods: the North/West Cluster and the South Cluster. Each one of the clusters was constituted by 
high schools and elementary schools, community-based organizations, parents, students, and business organiza- 
tions. Each had a Cluster Leadership Team to assure that the planning process was proceeding effectively and that 
a mechanism was in place to monitor the steps, expand the collaboration, participate in an environmental scan, and 
evaluate the ENLACE pipeline during the planning phase. 

ATransition Council was also constituted of ENLACE partners from the admission offices and transfer centers of two- 
year and four-year institutions; Latino business leaders; and guidance and counseling staff from high schools, 
alternative high schools, and community educational service centers, such as ASPIRA and LULAC. The council was 
to provide advice to partner schools and organizations, and implement and expand dual admission and articulation 
agreements between community colleges and mainstream institutions. The Transition Council was encouraged to 
work closely with partners in both clusters. 

The Chicago ENLACE Partnership made substantial progress toward building broad participation in decision making 
through community meetings, the formation of committees, and “inclusiveness interviews” of stakeholders in the 
community, soliciting their ideas and their participation in the initiative. During its planning phase, the Chicago 
ENLACE Partnership was composed of thirty-four partners in the following seven sub-groups: five elementary public 
schools, three public high schools, four alternative high schools, two business organizations, six community or junior 
colleges, one university, and thirteen community-based organizations. 

The main focus of the ENLACE Phase I planning grant was to "formulate creative approaches and models to prevent 
high school drop out and support Latinofa] youth to enter and complete college.” The identification of the barriers, 
strategies, and initiatives to address them were the result of more than sixty meetings with hundreds of individuals 
and stakeholders, many of whom experienced barriers in the pipeline, including former dropouts and pushouts from 
public schools. Latino staff from partner CBO’s, alternative high schools, elementary and secondary schools, 
community colleges, and universities worked on six working teams in developing the implementation plan. The 
environmental scan involved seven intensive focus groups with Latino parents in the various Latino neighborhoods 
of Chicago and ah equally focused input from students. 

The completion of Phase I of the Chicago ENLACE Partnership encoiiraged NEIU to secure funding for the ENLACE 
Phase II: Implementation. Funding for four years will be granted to ten of the eighteen Phase I Hispanic-serving 
institutions. Grantees during Phase II are expected to provide evidence of a “coalition’s ability to leverage change in 
higher education for Hispanics” (W.K. Kellogg Foundation, 1999. An Invitation for Proposals for ENLACE]. 






To ensure the sustainability of ENLACE beyond the five years of funding, the Chicago ENLACE Partnership needs to 
link its local activities to other local and national educational agendas. 
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Northeastern Illinois University is on the right track to respond to the challenges of the twenty-first century and to 
respond to the national goals for improving the educational achievement of Hispanic students; and more specifically 
to Goal 5-increasing postsecondary completion-and to benchmarking to this goal: doubling the percentage of 
Hispanic Americans who earn associate’s and bachelor’s degrees by 2010. 



Joaquin Villegas is Assistant Professor at Northeastern Illinois University in Chicago. 

Santos Rivera is Assistant to the President at Northeastern Illinois University in Chicago. 

Estela Lopez is Provost and Vice President for Academic Affairs at Northeastern Illinois University in Chicago. 
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The K-12 Kansas City Missouri 
School District- 
Higher Education Partnership: 
Where the Rubber Meets the Road 



Robert B. Stein 
Madeleine Kernen 



Introduction 

Urban school systems throughout the country are plagued with several challenges, including high turnover of 
personnel [administrators and teachers] and low student performance. As a result, the economic well-being of 
individuals, states, and the nation are at risk. Higher education’s tendency to deny partial responsibility for urban 
school problems or to lay total blame on elementary and secondary school systems for low student performance and 
low college-attending rates underscores a lack of understanding about the interdependence between higher 
education and K- 1 2 school districts. Increased emphasis on the connection between the level of education completed 
and an individual’s lifetime earning potential has focused a spotlight on school/college partnerships, especially within 
urban regions. 

Too often research and professional preparation agendas of college faculty and departments have defined the 
content of school/college partnerships. When this occurs at the expense of the educational needs of students and 
communities, the integrity of the process is at risk. Furthermore, while the unbridled competition driven by institutional 
aspirations and faculty career interests contributes to a rich diversity of postsecondary options, it can also result in 
a sporadic display of school/college programs. From the collective perspective of a K-12 school district, school/ 
college programs may appear to lack coordination, concentration, and focus. Under these conditions, while individual 
students may be served well, the system as a whole may fail. Without an agreed-upon vision and a joint commitment 
between K-12 and higher education to systemically and collectively improve all student achievement at all 
educational levels, a shared responsibility for integrity in the educational process is missing. Repetition of 
unsuccessful behavior patterns in K-l 2/higher education relationships should be unacceptable. Experiences of one 
Missouri school district in forming new partnerships with colleges and universities serve as a case study to inform 
educators interested in designing new approaches to school/college relationships. 



The Kansas City Missouri-Higher Education Partnership (KCMSD-HEP) 

Faced with the typical problems of an urban school district and a desegregation battle that lasted well over 25 years, 
the Kansas City Missouri School District [KCMSD] has been in a crisis mode for several years. Ongoing lawsuits and 
a cantankerous and divisive local Board of Education supportive of the status quo have led to great difficulty in 
implementing change-a situation that is also largely due to an alarming lack of continuity in leadership. 

On August 1, 1999, the current superintendent, Benjamin Demps, Jr., was brought into the district from outside the 
educational establishment [the military]. Shortly after his appointment, the Missouri Department of Elementary and 
Secondary Education announced that the KCMSD would likely lose its accreditation. Mr. Demps’ fresh perspective, 
energy, and, most importantly, tenacity have served him well in working with a festering school district to try to foster 
a new way of doing business. The loss of state accreditation in May 2000 forced all stakeholders to face serious 
O lortcomings in the district. While disagreements abound about which way to proceed and who is to blame, all 
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operations have been called into question, including the relationships between the district and external groups, such 
as colleges and universities, the Missouri Department of Elementary and Secondary Education [DESE], and the 
Coordinating Board for Higher Education. Under the leadership of the new superintendent, the KCMSD is seeking 
innovative approaches to regain state accreditation as well as to respond to court-ordered goals. 

Several forces converged, thereby starting the process for the formation of the new partnership between the district 
and its higher education partners. The commitment of the new superintendent to experiment, the assertiveness of 
a regional university some distance from the district, and Missouri’s K-16 agenda have each placed a spotlight on 
past practices concerning school/college relationships. Despite calls for more collaboration throughout Missouri’s 
system of higher education, local Kansas City-area higher education institutions felt they had claim in serving the 
KCMSD. Not only were several programs already in place, but their close geographical proximity seemed to assure 
their jurisdiction. A regional university located in a rural setting some 100 miles from Kansas City expressed a strong 
interest in developing an urban residential internship for its students, whereby they could challenge assumptions 
about past practices. Although the goal was worthy, it had the potential to create tensions. Both the regional university 
and the KCMSD were strongly encouraged to involve greater numbers of colleges and universities in this and other 
projects. As a result, an expanded conversation about new initiatives within the district emerged. 

These exchanges led to several informal discussions about the value of forming a consortium. Despite the best of 
intentions, however, colleges and universities found it difficult to move quickly. The high stakes associated with the 
loss of state accreditation sounded the district’s alarm. On a very short one-week notice, Superintendent Demps 
called presidents and chancellors of all higher education institutions with an interest in serving the district to a 
meeting. Despite the short notice, several institutions attended this historic meeting on June 30, 2000, formally 
beginning the exploration of a new partnership formation. The message from the superintendent was quite clear. He 
was moving forward on a fast track with whatever institutions wanted to participate. The clock was already ticking, 
and he did not have the luxury of working with institutions individually. While underscoring the wealth of expertise 
within higher education, he decried the lack of synergism throughout the system and across educational sectors. At 
the second meeting, held in July 2000, institutions committed to work collaboratively with each other and with the 
district. As first steps, institutions were asked to inventory all ongoing projects with the district. The KCMSD issued 
a Request for Information, summarizing its needs: a listserv was established, and draft principles for the consortium 
were developed. The intent of these activities was to identify and better understand the existing opportunities for 
addressing the complex and multiple challenges faced by the district. 

The name “Kansas City Missouri School District-Higher Education Partnership" (KCMSD-HEP] was chosen to give 
the group an identity. In addition to the district, partner members include seven public four-year institutions, the 
University of Missouri system, three independent four-year institutions, one community college, and the state 
commissioners of education and higher education. The new partnership was officially announced on October 30, 
2000, at a press conference in Kansas City, generating extensive media coverage. 

A primary objective of the KCMSD-HEP is to establish a more meaningful relationship among all members in 
supporting the agreed-upon goal of improving student performance. By pooling resources and coordinating 
communication, the KCMSD-HEP intends to improve the effective and timely delivery of services provided by higher 
education to the students, faculty, and administrators of the KCMSD. With the district’s needs as the primary force, 
partners are seeking new collaborative approaches for the delivery of service. While ongoing projects are not being 
eliminated, KCMSD-HEP is helping to refocus, coordinate, and prioritize school/college partnership activities in the 
district. New projects range from Saturday school (to improve student performance] and an urban internship year 
in the KCMSD to alternative pathways to become teachers, ways to increase teacher retention, and ways to enhance 
the managerial skills of district personnel. It is imperative that these needs be fulfilled rapidly and directly in a 
concerted and coherent way. Moreover, the KCMSD must concentrate all of its training efforts for district teachers 
and administrators on those educational needs that directly impact its accreditation status and compliance with court 
orders. In sharing responsibilities and in promoting and implementing new designs in support of the KCMSD, the 
integrity of the educational system can truly be showcased. 



Challenges Faced by the KCMSD-HEP 

□ Scope of Collaboration 




Initially, new partnerships must decide about questions of scope and magnitude. While some members want to 
proceed cautiously and are more comfortable with limiting the extent of collaboration to a single project, others 
see the benefit of fully integrating new ways of communicating and doing business into everyday routine. 
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The KCMSD-HEP envisions a coordinated approach in which all members have access to the same information 
and may participate collectively in evaluating the available resources and identifying the deliverers of service. 

□ Setting the Pace 

Collaboration requires integrating the perspectives of others. Early on, participants addressed the speed at which 
they would proceed. It was evident from the beginning that time perceptions varied greatly among KCMSD 
members. While short- and long-term goals have been established and the KCMSD-HEP is using a committee 
structure to identify solutions for the most immediate problems, new projects are being launched at a rate slower 
than some would like. With a commitment to move at a faster pace, institutions are raising money to hire a 
coordinator who will have responsibility for keeping KCMSD projects moving forward. 

□ Seeking a Common Language 

Partners soon discovered that language can serve as a barrier when forging a new relationship. Without clear 
definitions of common terms and acronyms, communication was threatened. In working across institutional 
boundaries and educational sectors, KCMSD members have focused attention on the language used in formal and 
informal interactions to ensure inclusiveness and common understanding. For example, the term faculty ; without 
a qualifier, may be interpreted to mean higher education personnel only. Educators, or K-1 6 faculty, more directly 
implies personnel from both sectors. Most recently, a briefing was held to provide greater understanding among 
higher education partners not only with the content and standards but also with the language and labels involved 
in the Missouri School Improvement Program [MoSIP], which is the state’s program for accrediting school districts. 

□ Overcoming Competition 

While interest in consortia arrangements is on the rise, colleges and universities have had a long history of 
competing with each other. The call for greater collaboration is often accompanied by accusations that loss of 
autonomy and hidden agendas will negatively affect the future of a particular college or university. Although 
insufficient for changing competitive patterns, the public agreement signed by the presidents and chancellors has 
sent a strong message to college and university faculty and administrators that partnering with colleagues at other 
institutions in the design of school/college projects should be considered normative rather than the exception. 

Lessons Learned 

□ Structuring Inclusiveness 

Genuine interest in using a consortia arrangement to restructuring relationships between colleges/universities 
and K-12 schools requires constant attention. Not only are colleges and universities required to blur institutional 
boundaries in meeting the challenges, K-12 partners are expected to have a major say in the design of projects 
that involve their schools, their students, and their staff. Engaging representatives from all consortia members in 
the development of new initiatives is necessary to increase ownership in potential activities. No matter how worthy, 
a new initiative developed “behind the scenes” by a subgroup of consortia members can threaten a group’s sense 
of trust and reinforce a return to competitive behavior. 

□ The Use of Space 

An overlooked elementthat affects relationships between and among consortia members involves the use of space 
during consortia meetings. Where meetings take place and room setups are symbolic messages about the roles 
and responsibilities of partners. Alternating locations and hosts for meetings has reinforced a sense of shared 
responsibility among partners. Controlling seating patterns through the use of placards has ensured that the 
superintendent and his staff, who are placed at the head of the table, are perceived as driving the conversation 
about the needs of the district. In addition, since institutions vary in the number of personnel they bring to the 
meeting, permitting each institution only one seat at the main table reinforces the perception of equity. 

□ Breaking with the Past 

The request to change behavior patterns creates an image that past practices were rampant with problems. When 
individuals whose behaviors require change become defensive, they often feel a need to defend past practices. 
By acknowledging positive results from past practices and emphasizing structural changes as a driver for change, 
the superintendent is able to reduce defensiveness on the part of colleges and universities. 
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□ Low-Stakes Information Gathering 

During the early stages in the formation of the KCMSD-HEP, it was assumed that a Request for Proposals (RFP) 
process would be used bythedistricttogetits needs met. It was envisioned that institutions or groups of institutions 
would respond to the RFP and that the best proposals) would be chosen. Because this process promoted 
unnecessary competition among partner members, the approach was shelved, and a Request for Information (RFI) 
process was used instead. The RFI, in contrast to the RFP, is structured as a low-stakes process. As a result, 
information about the KCMSD’s needs and higher education’s available resources and ideas to meet those needs 
has surfaced without forcing a high-stakes decision. 



Conclusion 

The KCMSD-HEP is a new partnership between a local urban school district and its higher education partners. A sense 
of urgency experienced by the district resulted in the superintendent’s challenging higher education institutions to 
work collaboratively with the district. The Memorandum of Understanding, signed in October 2000, provides a 
foundation for changing the traditional behavior patterns that tended to foster competition among institutions; 
promote higher education-driven projects; and reinforce individual rather than systemic change. Members of the 
KCMSD-HEP are using baseline data to drive discussions about the district’s need to improve student performance 
and to design new innovative approaches to sustain and enhance a quality teaching workforce in the district. As a 
result of intensive dialogue and information exchange, new understandings are emerging about the interdependence 
between K-12 and higher education systems. The KCMSD is still in a crisis mode for its survival. Through the work 
of KCMSD-HEP, the school district, college and university partners, and the state departments of education and 
higher education have accepted joint responsibility for the crisis and are working collaboratively to identify viable 
solutions. 
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The Dual-Credit Explosion 
in Community Colleges 



Hans A. Andrews 
Joseph Cipfl 
Robert Marshall 



One of the most exciting and meaningful programs of cooperation between the secondary schools and the 
community and senior colleges in America in recent years has been the provision of dual-credit and 
concurrent enrollment options for junior and senior high school students. 

In Illinois the dual-credit program is being offered as part of the answer and solution to the Affordability 
Committees' recommendations on how to contain the costs of higher education. This statewide committee 
was also seriously concerned about the large number of students taking from 4.5 to 5.5 years to complete a 
baccalaureate degree. 

The following is a summary ofthe key findings ofthe study. The next paper, “Outcomes of Dual-Credit for Students," 
outlines the several types of models colleges may wish to set up with secondary schools and their students. 



One ofthe newest and largest educational movements in Illinois and across the country is that of dual-credit enrollments 
for secondary school students. In 2000, research conducted at 48 public community colleges found a 240 percent growth 
in this program in Illinois community colleges. In some states this program is called concurrent education . 

All 48 of the public community colleges responded to a survey to determine how rapidly this program was expanding 
and to determine the issues and the type of enrollments taking place at the beginning of the new century. Colleges 
reported that a total of 290 secondary schools were participating during the 1999-2000 academic year. This number 
was up from 120 secondary schools two years earlier. Why the large increase? 

The Illinois Community College Board, with strong encouragement from the 48 colleges, made a funding change in 
1996. It allowed the community colleges to submit credit hour grants even though the secondary schools were 
claiming the students as part of their average daily attendance. This means that colleges and the secondary schools 
no longer suffer a financial loss in the dual-credit program. With this financial barrier removed, the colleges have 
reached out to include as many secondary schools as wished to be a part of the program. 

The dual-credit program received another push in Illinois when an affordability study committee raised issues about 
the length of time it was taking many students to get to a baccalaureate degree. The cost of such a degree was also 
mentioned as a problem. One ofthe recommendations was to expand opportunities to secondary school students who 
could benefit from accelerated programs. 



Dual-Credit Defined 



Dual-credit is defined as college courses offered to secondary school students who receive both college credit 
and credit toward secondary school graduation. 
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Dual-credit classes were found to be offered in three major areas: (1) transfer courses that lead to the baccalaureate 
degree; [2) technical courses that lead toward an applied associate degree or a college certificate in the technical 
field; and [3) vocational courses that can also lead to the applied associate degree or a technical certificate. 
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The Dual-Credit Option for Students 

Secondary school administrators and counselors are using the dual-credit option to challenge juniors and seniors 
in high school. The program carries college credit, very much like the Advanced Placement (A.P.) program that a 
significant number of secondary schools have had for a number of years. 

Talented students no longer have to wait to finish high school before starting college-level work. For example, 
Marshall and Andrews identified a program at Marquette High School in Illinois that allows students the option of 
three or six dual credits a semester. The local community college provides the courses at the high school during the 
regular school day. Some students complete 24 credits during the two years, and by taking one summer session they 
can start their first full year of college as sophomores! 



Course Offerings 

Students who have special talents in art, computers, music, automotive, and other course areas are also enrolling in 
these programs across Illinois. The 2000 study in Illinois showed the following array of course options being the most 
consistently offered during 1999-2000: 

Most offered dual-credit courses for transfer college credit: 



English 101 22 colleges 

Psychology, Introductory 17 colleges 

Mathematics 15 colleges 

History, U. S 11 colleges 

Calculus 10 colleges 

Sociology, Introductory 7 colleges 

Most offered dual-credit courses for technical college credit: 

Cisco Networking 13 colleges 

Mechanical Technology 9 colleges 

Computer Information Systems 6 colleges 

Most offered dual-credit courses for vocational college credit: 

Automotive 14 colleges 

Nursing Assistant 4 colleges 

Welding 4 colleges 

Electronics; Engineering Technology; 

Cosmetology 3 colleges 



There were more than 80 courses listed in the summary of the surveys. 



Quality Safeguards 

The survey asked all colleges about their concerns relative to quality in the program. The top two responses were 

(1) placement testing to assure proper placement (and readiness) for courses; and (2) having faculty in the program 
who meet the qualifications and competencies required by the college to teach the dual-credit courses. In most cases, 
respondents mentioned that the same qualifications for full-time and part-time faculty were expected. Colleges used 
both full-time and secondary-school-qualified faculty in their programs. 

Some of the other safeguards mentioned were: (1) having the same course syllabus as courses offered on-campus; 

(2) using the same textbooks as on-campus; (3) having department chairs observe the class(es) once a semester; 
(4) using secondary school counselor recommendations; and (5) using student evaluations. 



Outlook 

Colleges were asked about the future for this program. Most of the responses were very positive. Respondents called 
the program “a win-win program,” “here to stay,” and indicated that “the senior year schedule will look more like a 
college schedule,” and “it offers stronger communications between secondary and post-secondary institutions." 
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Concerns 

Concerns centered around maintaining the quality and integrity of the classes taught for the secondary school 
students. There was some concern that college faculty might lose some of their future enrollments. Transportation 
of students and union concerns in both college and secondary schools were identified. Finally, the ineligibility of high 
school students for financial aid and resistance of some secondary schools to the program were also mentioned as 
concerns. These concerns were fairly isolated, and none were mentioned by more than two or three colleges. 



Summary 

Illinois community colleges have experienced a 240 percent increase in the dual-credit program over a two-year 
period. A funding change made a major impact, and the number of secondary schools with students enrolled jumped 
from 120 to 290. 

The prognosis for this program is strong. Growth is expected in the coming years. Quality issues were the main 
concerns. It appears that these issues will be addressed, which will make this a most acceptable program for students, 
parents, secondary schools, and the universities and colleges to which students will transfer their credits. 

This outline focused on the Illinois dual-credit system. There are now more than 30 states that have been identified 
as having dual-credit and concurrent enrollment plans. 
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Appendix 

Outcomes of Dual-Credit for Students 



Hans A. Andrews 
Robert Marshall 
Jackie Davis 



The dual-credit program movement in American community colleges and second- 
ary schools has expanded greatly in the past four or five years. In Illinois there was 
a 240% increase in the number of secondary schools taking part in this program 
from 1997-1998 to 1999-2000. In some states the program is called concurrent 
enrollment. 

The importance of the dual-credit program is being documented in a number of 
states where it has existed for several years. It provides an exceptional option to 
secondary schools that wish to offer challenging programs to their honor students 
and to students who need to obtain technical and vocational background for the 
workforce. 



Definition 

Dual-credit courses, as 
defined here, are college 
courses offered to 
secondary school students 
who enroll and receive 
both college credit and 
credit toward secondary 
school graduation. 



Outcomes for Students 

The dual-credit program offers several outcomes for secondary school students: 

o An opportunity to enroll in advanced college-level coursework while still in high school 

o An opportunity to gain marketable technical or vocational skills not offered by the secondary school 

o An opportunity to earn up to one semester of college work prior to (or immediately following) high school 

graduation 

o An opportunity to earn up to one year of college work prior to 
graduation 

o An opportunity to earn up to two years of college work prior to 
graduation 

□ Dual-Credit Model I 

Obtaining one semester of college work concurrent with high school graduation. 

A student would enroll in one course per semester for each of the four semesters of the junior and senior years 
and one summer course after the junior or senior year. 

15 credit hours: one semester 



(or immediately following) high school 
(or immediately following) high school 
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□ Dual-Credit Model II 

Obtaining one year of college work concurrent with high school graduation. 

A student would enroll in two courses per semester for each of the four semesters of the junior and senior years 
and two summer courses either after the junior or senior year. 

30-32 credit hours: one year 
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□ Dual-Credit Model III 

Obtaining two years of college (Associate Degree ) concurrent with high school graduation. 

A student would enroll in four or five courses per semester for each of the four semesters of the junior and senior 
years and two summer courses after both the junior and senior years. 

60-64 credit hours: 2 years (Associate Degree) 

□ Dual-Credit Model IV 

Obtaining certification in a marketable technical area: e.g., CISCO Networking Specialist; Microsoft Certification, etc. 
A student would enroll in one course per semester for each of four semesters of the junior or senior year. 

12 credit hours: Cisco Certification, for example 

□ Dual-Credit Model V 

Obtaining courseworkin technical or vocational classes that leads toward a certificate or technical applied Associate 
Degree at a community college or into an entry level job. 

A student would be enrolled at the college or secondary school to obtain credit in any of the college’s technical 
or vocational offerings. 

Credit: variable 



Making It Work! 

The dual-credit course option is a fast-expanding program between community colleges and secondary schools. In 
a number of states, universities and other four-year colleges are also a part of the delivery of dual-credit options to 
students. Some important things need to happen to create a quality dual-credit program: 

o Agreements between community colleges and a secondary schools need to be worked out 

o Faculty utilized in the dual-credit program need to be of the highest quality and must meet the hiring 
requirements of the community college, use college syllabus, and use college textbooks 

o Students and parents need to learn how these courses articulate into the degree programs at community 

colleges and senior colleges or into employment 

o Counselors at both the secondary school and college need to work together to assure students and parents 
of proper placement into the college coursework 



For more information on this program, please contact: 



Dr. Hans A. Andrews 
Olney Central College 
Olney, IL 62450 
Fax: 618-392-5212 
E-mail: andrewsh@iecc.cc.il.us 



Dr. Robert Marshall 

Illinois Valley Community College 

Oglesby, IL 61348 

Fax: 815-224-3033 

E-mail: marshall@ivcc.edu 



Mr. Jackie Davis 
Olney Central College 
Olney, IL 62450 
Fax: 618-392-5212 
E-mail: davisj@iecc.cc.il.us 
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Bridges to Rural Nebraska: 

An Online Model to Address 
Problems in Early Entry Programming 
(College Courses in High Schools) 



Carl 0. Ellis 
Andrew Elkins 

with input from 
W. Snyder 
Joe Kincaid 
D. J. Weitzenkamp 



In many states, colleges are providing high school students with opportunities to earn college credit during their junior 
and senior years. These courses are offered in various forms and under different names, but they are usually taught 
in the high school classroom and involve some form of collaboration between secondary teachers and college 
professors as well as eligibility criteria for high school students and teachers. 

The programs are increasingly popular. Secondary administrators love them because they can tell parents that their 
schools offer college credit; students flock to them because they can get an early start on their college careers; parents 
are proud that their children are “advanced"; and college administrators see profit. 

However, the growing popularity of these programs has taxed the instructional resources of many colleges and raised 
disturbing questions about quality. Peru State College, in Peru, Nebraska, is experiencing those same problems: the 
College has not handled the growth in its Early Entry program very well over the last five years. 



Problems 

The problems can be categorized as follows: 

[1] High school student qualifications 

[2] Secondary teacher qualifications 

[3] High school classroom environment 

[4] College faculty ownership, control, and review of the early entry courses. 

CD High School Student Qualifications 

The Nebraska Commission on Post-Secondary Education's “Guidelines for the Academic Transfer of College 
Courses Delivered in High Schools to High School Students" establishes six requirements that all students in the 
college-credit classroom must meet Students must: 
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a) be recommended by the high school academic staff; 

b) meet the prerequisites of the course; 

c) have completed courses that meet the related high school subject matter admission requirements at the 
postsecondary institution where they intend to transfer the college credit; 

d) have a cumulative grade point average of B, its equivalent, or better, or a composite ACT score of 22; 

e) rank in the upper third of their class academically; and 

f) be juniors or seniors. 

Exceptions to requirement e may be made by the college official supervising the academic unit responsible for the 
delivery of the course. 

The problem is that in order to increase their enrollments in these courses, high schools have been asking for 
numerous exceptions to the above qualifications, most often using teacher recommendations [a) to get around 
other requirements, especially d and e. 

The college, in order to increase the flow of tuition dollars and to keep its program alive, has too frequently, even 
routinely, approved the requests for exceptions, taking full advantage of the provision that a “college official” may 
grant exceptions. 

(2) Secondary Teacher Qualifications 

According to the North Central Association’s Handbook of Accreditation (2 nd ed., 1997), Chapter 3, “General 
Institutional Requirements,” 

faculty teaching general education courses hold graduate degrees that include substantial study [typically 
a minimum of 1 8 semester hours at the graduate level) appropriate to the academic field in which they are 
teaching. [24) 

This rule is relevant because the bulk of the early entry courses are general education courses. However, secondary 
faculty in early entry have been approved routinely if they hold a master’s degree of any sort, usually one in 
education, even without the required "appropriate” 18 graduate hours. 

(3) High School Classroom Environment 

Small schools are the rule ratherthan the exception in depopulated rural Nebraska, and too often in these schools, 
students who are taking, for example, a history course for high school credit were seated and taught in the same 
room atthe same time by the same teacher as students taking a history course for college credit. Obvious questions 
of quality control and legitimacy come to mind when high school and college students are mixed together and 
taught at once. Is a college text being used? Are college-level tests being administered? Is a syllabus appropriate 
for college credit being followed? 

(4) College Faculty Ownership, Control, and Review of the Early Entry Courses 

Referring again to the NCA Handbook it is “essential” that “faculty have ownership and control over the general 
education curriculum” and that “faculty systematically and comprehensively review the general education 
curriculum.” [24) 

The Handbook goes on the stress that "the fact that the general education faculty might not be on the campus does 
not diminish the need forthe institution to ‘own’ the general education program” [46), a direct shot atthe tendency 
of colleges to establish early entry courses in secondary schools and then ignore them as long as the classes 
continue to make a profit and the next NCA visit is several years down the road. 



Solutions 



Peru State College was faced with all these problems. Several options presented themselves as solutions: faculty 
mentors, interactive television, and online. The only affordable and practical solution seemed to be an online model 
that, when fully implemented, will be a true bridge between the college and the secondary classroom. 
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The online model is relatively simple: 

o On-campus, fully qualified faculty are paid a stipend by the college to develop early entry courses on the Peru 
State Blackboard server, thus establishing ownership and control of the courses. Faculty are also provided 
technical instruction and support as they create the classes, as recommended by the Draft Statement of the 
Regional Accrediting Commissions on the Evaluation of Electronically Offered Degree and Certificate 
Programs [September 2000: 3]. 

o When the courses are offered to the high schools as early entry courses to replace the current model, the Peru 
State faculty will serve as the faculty of record and the secondary teacher will act as a facilitator. 

The model, although simple, addresses all of the above problems: 

o Qualified college faculty write, review, and revise the course content as well as randomly check the students’ 
performance and production. 

o The classroom environment problem is eliminated because the early entry courses now exist in a virtual 
environment. 

o High school students no longer are being admitted to the program with exceptions. 

Therefore, the online model: 

o Gives the college ownership 

o Ensures that the early entry courses are consistent with the college’s mission 

o Provides more opportunity to students, who now can take the classes at flexible hours and who are no longer 

limited in class selection by the availability in their high school of qualified teachers 

o Keeps parents, administrators, and accrediting bodies happy. 

Technology in this case clearly bridges two instructional worlds and results in a much improved instructional offering. 



Carl 0. Ellis is Associate Vice President for Technology and Extended Campus at Peru State College in Peru, Nebraska . 
Andrew Elkins is Dean of Arts and Sciences at Peru State College in Peru, Nebraska. 
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A Small Rural College 
Collaborates with an 
Elementary School to Create a 
Professional Development School 

Beth Musser 
Earl Nicodemus 

Description of Partnership Schools 

West Liberty State College (WLSC) is located in the narrow northern panhandle of West Virginia between 
southwestern Pennsylvania and eastern Ohio. Established as West Liberty Academy in 1 837 C25 years before the state 
was admitted to the Union), it was created to respond to the need for higher educational opportunities west of the 
Appalachian Ridge. Currently, the college population is 2600. 

The School of Education enrolls 900 students (35 percent). Of these, approximately 500 are elementary education 
majors. Program accreditation is provided by the National Council for Accreditation of Teacher Education (NCATE) 
and the West Virginia Department of Education. 

Madison Elementary School is located on an island in the Ohio River. Most of the 400 students (PK-5) live on the island. 
Madison Elementary is a Title I school receiving additional funding to provide reading and math support to all children 
enrolled in the school. Approximately 75 percent of the students at Madison qualify for the reduced-lunch program. 
Because of the commitment of school faculty and staff, Madison Elementary is a National Blue Ribbon School of 
Excellence and has won other awards for innovative practices. 



History of the Partnership 

The Professional Development School [PDS] shows the greatest promise in education reform because it aims to 
advance teacher education reforms and school reforms simultaneously. The promise of PDS was the impetus for 
WLSC and Madison to collaborate in the fall of 1998. The Madison Local School Improvement Council (LSIC) and the 
Ohio County School Board approved the partnership with WLSC in August 1998. This relationship formalized and 
extended the work WLSC faculty had established through a clinical literacy experience for teacher candidates. 

Funding for the first three years of the project 0998-2001} was provided by a grant from Dewitt-Wallace/Reader’s 
Digest and NCATE. The WLSC/Madison PDS was selected as one of 19 partnerships nationwide to field-test PDS 
standards. 



Best Practices Associated with the West Liberty State College/Madison PDS 

The following concepts are part of the partnership: 

o College instructors collaborate with the principal and teachers to select and assign student teachers to 
particular teachers. 
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o The college supervisor, cooperating teacher, and student teacher hold regular conferences [triads). 

o Both college and school faculty teach preservice teachers in the PDS program. A curriculum and methods 
course is taught on-site at Madison by a Madison staff member and a college faculty member. The school 
principal teaches an educational psychology course at the college. Reading Recovery teachers from Madison 
meet with junior literacy students. The school counselor teaches a course at the college. 

o A continuum of placement helps college students become a part of the community of learners. College 
students are placed as observers-participants in their freshman and sophomore years. Since 1993, all 
elementary majors have participated in a literacy [reading and language arts) program called Building Bridges. 
The Juniors and Seniors observe third grade teachers and create mini-lessons for small groups of third 
graders. 

o By the time the college students are student teachers, they have spent portions of two or three semesters in 
the school and have become part of the school/learning community. 

o Inquiry and action research are elements of the partnership. Teacher candidates and master teachers 
collaborate to facilitate student learning and carry out the goals of this partnership. 



Governance 

The PDS Council consists of the building principal, a parent, a teacher, the WLSC Dean of Education, a WLSC 
education professor, and a central office administrator. The Local School Improvement Committee CLSIC) is the 
primary decision-making committee, and the PDS Council is represented on this committee. 

0 The four functions of a PDS. The mission of a PDS is four fold: teacher preparation, staff development 
practice-based inquiry, and student learning. The WLSC/Madison PDS has sponsored in-service programs 
for both preservice teachers and faculty. Action-research projects are conducted by student teachers. A 
faculty member received several small grants that she used to survey the attitudes of PDS student teachers 
and non-PDS student teachers. 

The partnership research has verified that student achievement has improved over the last three years. More 
research needs to be conducted in order to assess the impact of the partnership on student learning. 

0 The NCATE PDS standards. There are five standards, each addressing a critical attribute of a PDS 
partnership: Learning Community; Accountability and Quality Assurance: Collaboration; Diversity; and 
Structures, Resources, and Roles. The PDS Standards are intended for self-assessment They are measures 
of process accountability, but they can also serve as tools for change and development 



Conclusion 

In the December 1999 issue of the Journal of Teacher Education, there is an article entitled “Paradise Unrealized, 
Teacher Educators and the Costs and Benefits of School/University Partnerships.” The authors state that “getting a 
partnership off the ground is relatively easy; sustaining it over a long period of time is not.” The WLSC/Madison PDS 
has found this to be true. 



Beth Musser is Interim Dean, School of Education, at West Liberty State College in West Liberty, West Virginia. 

Eari Nicodemus is Interim Chair of the Department of Professional Education at West Liberty State College in West 
Liberty, West Virginia. 
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The Century Wall: 
Balancing the Benefits of Diversity 
and the Values of Community 



Jan Fielder Ziegler 
Sandy Davis-Baltz 



Introduction 

The Century Wall was officially unveiled on July 4, 2000, to the cheers of an enthusiastic crowd eagerly awaiting their 
chance to finally see, touch, and commemorate this public monument that had been so long in the making and in 
which community involvement was so widespread. Before the last festive balloon had drifted out of sight, individuals 
in the audience moved forward within the circle of the three sections of wall to gaze more closely and to marvel at 
the faces of Americans who had helped to shape this nation in the twentieth century. They pointed to the faces of 
those easily recognized-'There’s FDR!” or “I see Oprah Winfrey!” and they noted with satisfaction, “I nominated him/ 
her!” They gazed also at unfamiliar faces, many of them Asian Americans, Hispanic Americans, African Americans, 
and Native Americans. They looked at the legend at the bottom of each wall to check the name of the person below 
Walt Disney or to the left of John F. Kennedy. When neither the face nor the name was recognizable, Century Wall 
visitors thumbed through booklets to learn the identity of the person, and why he/she had been selected as one of 
the hundred individuals depicted on this unique structure. 



Background 

Black River Technical College (BRTCJ was one of forty two-year colleges nationwide selected in 1996 to participate 
in the “Exploring America’s Communities: In Quest of Common Ground” project. Sponsored by the National 
Endowment for the Humanities, the American Association of Community Colleges, the Community College 
Humanities Association, and Phi Theta Kappa, this project was designed to enhance the teaching of history and 
literature at the country’s two-year colleges. BRTC’s winning proposal included a plan to involve not only the 
institution, but also the entire community in a project that would promote awareness of the beauty and strength of 
this nation in the twentieth century, including the incredibly important diversity of our people. Because ours is a 
community-and an institution-where all minorities combined total less than two percent of our populace, we 
perceive this to be a particularly compelling need. 

The project called for community-wide discussion and study to select 1 00 individuals who, in all the rich tapestry that 
reflects the people of this nation, had made a difference in shaping twentieth century America. The faces of these 
individuals, according to the original proposal, would be depicted on a large mural on a building in the historic 
downtown area. Although the plan fora mural eventually evolved, as did the location of the wall, the seminal concept 
remained constant: This would be a community undertaking to engage students, faculty, staff, and those outside the 
institution itself in a consideration of those who had an impact on the nation we have become as we end one century 
and begin the new one. The wall would be a celebration of America and the nation’s endless human resources, 
including those from its diverse ethnic groups in the twentieth century. 



ERTC 



Simultaneously, BRTC English and history faculty were working with the national project organizers and a specially 
appointed mentor to implement not only the wall component of our plan, but also to undertake extensive study and 
reading and attend national conferences. As a part of the project, we were exposed to current literature and ideas 
on infusing our literature and history curriculum with previously omitted writers and themes that better reflected the 
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stories and the histories of all America’s people, particularly those previously distorted or marginalized. These 
activities complemented our ongoing wall development by increasing our understanding and knowledge-filling in 
gaps that even we, as educators, had to confront. 



Development of the Project 

The task we faced was a sobering one: Who were these 100 people to be? How would they be selected from the 
thousands of individuals who had written the lines of the country’s history in the past 100 years? Who would be so 
presumptuous as to make the decision? And how would we ensure that the final outcome would reflect the project’s 
goals of inclusion of lesser-known, lesser-appreciated Americans? 

To decide whose faces should be considered for inclusion on the wall, the Century Wall Committee members and 
BRTC English and history faculty began researching on their own while concurrently soliciting and receiving 
suggestions from individuals all over northeast Arkansas. Names of famous and not-so-famous Americans arrived 
in letters from historians at Arkansas State University and other institutions of higher education. We presented 
programs at the local Rotary, Kiwanis, and Women’s Clubs, and received their nominations. Names came in from 
history buffs in nearby towns, from public school students, and from BRTC faculty and staff. Nominations were 
submitted through editorial columns in local newspapers. Names were submitted by men, women, and children of 
all ages at our booths at the County Fair, Maynard Pioneers Days, and the Old Davidsonville State Park Pow-wow. 

The Century Wall was now more than just an idea; it was becoming real, long before anything tangible existed, made 
palpable by the slips of paper and letters and newsprint containing the names of hundreds of noteworthy Americans 
placed in nomination by the people of this and surrounding communities. All these names were added to the master 
list. Somewhere on this list were the hundred who would eventually end up, quite literally, etched in stone. 



The Century Wall Evolves 

Many obstacles, some of them seemingly unsurmountable, created fresh dilemmas on a weekly basis, or so it seemed 
to the Century Wall Committee. Months were spent wrestling with such issues as location-the ideal building would 
have open visibility but limited southern and western (read: excessive sunlight) exposure; surface-not every brick 
or stone would lend itself to a project of this exactness; life expectancy-depending on the elements, the mural might 
last seven to ten years, but it would look weatherworn after only a couple of years; and artists— lots of people wanted 
to do the project, but did not necessarily have answers for the obstacles. It was at this point that the mural idea 
metamorphosed into something far more durable and far more doable: We would create an actual wall using 
something-we did not yet know exactly what-and locate the structure, at the invitation of the Mayor, Parks 
Commission, and Chamber of Commerce, in the then-under-construction Black River Overlook Park where it would 
be in the heart of the community that created it. The possibilities instantly expanded, not only for realizing BRTC’s goal 
of raising community awareness of American diversity, but also for integrating this outcome with the wonderful 
economic benefit to the community of enhancing the city’s tourism and industrial recruitment efforts. 

If this project as it was evolving was more doable, it was also more expensive. But by now community interest was 
so substantial that we were able to forge a partnership to fund the project. Funding was pledged by the college’s Board 
of Trustees, the City of Pocahontas, Randolph County, and the community’s three financial institutions. 

A key step once the wall decision was made and projected funding secured was the selection of the college’s own 
art instructor as the Century Wall artist. This artist not only prepared for the actual drawing of the faces, once the 
selections were made, but also took the lead in moving the project from just an idea to reality. We finally chose a 
specially prepared concrete block produced in Wisconsin, and located an engraver in Minnesota who could actually 
transform pieces of art onto the blocks’ surface. 



Selection of the One Hundred Faces 

With assistance from the “Exploring America’s Communities" project directors, the Century Wall Committee 
developed a set of criteria by which those names on the master list would be measured and weighed. Among the 
criteria were a consideration of the extent to which an individual had positively impacted large numbers of Americans; 
whether he or she had to overcome personal or societal obstacles in the endeavor; the extent to which overall and 
enduring quality of life had been influenced; and a consideration of positive impact on shaping the nation's 
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acceptance of its diverse people. A committee of twelve (whose request to remain anonymous has been honored) 
sifted the names on the master list through the criteria filter, discussing, defending, discarding, voting, and eventually 
arriving at the names of those whose faces would comprise the Century Wall. Unable to narrow their selection to 
exactly 100, the group finally settled on a few multiple-face groups (the Wright Brothers, for example), resulting in 
more than 100 individuals, but producing 100 portrait entities overall. 



From Concept to Concrete 

The artist researched the photos in a tri-state region, relying on college libraries and, when no actual photographs 
existed, turning to crude drawings or Internet imagery. From these, the artist made initial pencil sketches. Using these 
sketches, she developed the placement, size, and view of each person, adjusting as necessary to produce an 
interesting yet easily readable design. Next came the actual ink drawings, which had to be extremely detailed yet 
scannable. To no one’s surprise, this project, a monument of a century, dictated both old-fashioned handmade 
artwork as well as the use of computer technology. Once the faces had been completed, numbered, and checked for 
accurate placement, much as if they were pieces in a giant puzzle, the drawings were shipped to the Minnesota 
engraver. He utilized computer technology to scan the images, which were then sandblasted onto the surface of the 
special blocks. Interestingly, high-tech laser engraving proved too inaccurate for this detailed piece of art. The register 
marks were critical for the faces to line up evenly so as not to produce a Picasso effect. After the stones were cut, 
the lines were then painted with a permanent black medium. The finished blocks were shipped to Pocahontas to await 
construction of the Century Wall. 

Amazingly, the three sections of wall, each of them approximately 7 by 1 5 feet, were completely constructed in less 
than a week. Throughout the construction, the townspeople watched with growing interest, because they had taken 
in the wall as a part of them. As soon as a section was completed, it was draped so as to reserve the final view for 
the July 4 unveiling. 



Well Worth the Wait 

From the moment of unveiling onward, the Century Wall has proven it has a place in this community. Together with 
the booklet, which is maintained in a “mailbox" on-site at the wall and is available free of charge to visitors, the Century 
Wall serves as a silent educator reminding all of us of the very real benefits of diversity and values of community. From 
its home on the west bank of the Black River alongside the main highway, the Century Wall stands facing the heart 
of the downtown Pocahontas area, beckoning those who pass by to stop, to think, to marvel, to enjoy, and to celebrate 
America. 



Jan Fielder Ziegler is Director of Planning and Assessment, Instructor-German, and Century Wall Project Chair, at Black 
River Technical College in Pocahontas, Arkansas. 

Sandy Davis-Baltz is Instructor of Fine Arts and Century Wall Artist at Black River Technical College in Pocahontas, 
Arkansas. 
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Economic Development: 
A Shared Responsibility 



Carolyn A. Taylor 
Sid Hudson 



Purpose 

Understanding the economic impact of colleges and universities has long been of interest to higher education 
administrators, policymakers, and public officials. Higher education institutions carefully walk the line between the 
pursuit of the traditional academic mission and the need for contemporary relevance-in other words, serving the 
common good. 

Colleges and universities are increasingly promoting themselves as institutions that have impact on regional, social, 
and demographic dynamics; enhance regional economic conditions; and contribute to regional income and to the 
regional tax base. This potential for positive economic and social impact is likely to be perceived positively by 
governing bodies elected or appointed to distribute limited state funds to all types of important state programs as well 
as to institutions of higher education. 

Universities and colleges play a valuable role in economic development, but that role is neither well defined nor easily 
understood. States and communities seeking to improve their economic fortunes are turning to universities to 
participate more fully in economic development. For their part, universities are promoting their own economic 
development agendas while trying to increase state and community support. 



An Oklahoma Partnership: Higher Education and Economic Development 

□ History 

Oklahoma’s economic history and the emerging global economic environment are at odds. Globally, economies 
are becoming more dependent on human capital as knowledge-based industries replace the old physical capital 
enterprises. However, Oklahoma lags behind other states CO in degree attainment and [2) in opportunities for 
research institutions and their faculty to participate in the private enterprise rewards for technology commercial- 
ization. The result is a shortage of high-paying jobs in Oklahoma. The Citizen’s Commission for the Future of 
Oklahoma Higher Education reported: “Except for a brief period during the oil boom in the late 1970’s and early 
1 980’s, Oklahoma’s per capita personal income has remained about 80 percent of the national average for half a 
century.” The report recommended a new course when it indicated that an increase in the state’s per capita 
education attainment and its ability to commercialize research outcomes would greatly affect the rate of future 
economic growth. 

□ Economic Development Grant Program 




During 1997-1998 the Oklahoma State Regents for Higher Education convened a panel of Oklahomans to help 
set a direction for Oklahoma higher education. The Citizen’s Commission on the Future of Higher Education 
developed a set of recommendations for the Oklahoma State Regents for Higher Education that included a 
recommendation that the State Regents should lead the way for greater direct involvement in economic 
development by Oklahoma higher education institutions. 



67 



Chapters New Designs in Higher Education. New Designs through National, State, and Community Initiatives / 61 



At the urging of the Citizen’s Commission on the Future of Higher Education, a three-part economic development 
initiative entitled 1998 Economic Development Plan was approved by the Oklahoma State Regents for Higher 
Education for use as a blueprint for more direct higher education assistance in economic development and is intended 
to “enhance the quality of life” in Oklahoma. The plan’s central goal is to establish a stronger, more flexible, and 
responsive Oklahoma economy by more closely linking higher education’s resources with the state’s businesses, local 
communities, and public state and local agencies. The plan will strengthen Oklahoma’s economic structure in three 
areas: 

1. Ability to cope with fast-changing workforce needs of high-paying job fields 

2. Propensity to develop areas of expertise in knowledge-based industries 

3. Capacity to transfer research outcomes to the workplace. 

In addition, it authorized a Request for Proposals to public higher education institutions entitled “Request for Grant 
Proposals: Working to Implement the Oklahoma State Regents’ Statewide Economic Development Plan.” Approxi- 
mately $3 million was allocated by the legislature to fund the grants for fiscal year 1999. An additional $3.4 million 
was awarded for FY 2000 grants and yet another $2.2 million allocated for FY2001. The proposal’s support works to: 

1 . Develop Oklahoma’s workforce 

2. Establish centers of intellectual excellence 

3. Commercialize research outcomes. 

This grant-making approach better positions institutions to lead the state’s economic and workforce development 
into the twenty-first century, creating, attracting, and holding high paying jobs. This program has made 28 awards 
for a total of $8.3 million. 

The program has: 

1. funded the creation of technology transfer offices at the University of Oklahoma and Oklahoma State 



2. helped institutions create academic programs in the areas of multimedia design, forensic computer science, 
biomedical/biotechnology, film and television industry, aerospace administration, workforce development, 
process technology, and information technology; 

3. stimulated multiple areas of institutional cooperation and created more business-higher education partner- 
ships. As an example, a $300,000 grant to the University of Oklahoma Health Sciences Center is building 
Oklahoma’s pharmaceutical industry; at Tulsa Community College, a $236,000 grant is implementing 
telecommunications technology and semiconductor manufacturing programs. These awards, plus many 
others, represent a major step in building Oklahoma’s economic development infrastructure by increasing 
competitiveness in key industries. 



Survey Findings 

Results from a recent statewide study documented the current role and activities of all Oklahoma higher education 
institutions and the extent to which institutions participate in selected economic development activities. All of the 
public institutions reported having a role in economic development. A large majority of the respondents stated that 
the major role was to provide employers with an educated workforce, fulfill the mission of the institution, provide 
leadership, build community partnerships, conduct research and share technology, assist with business development 
and share faculty expertise with the business community. As reported, over 90 percent of institutions surveyed noted 
increasing institutional activity in economic development over the preceding ten years. The findings suggest that 
institutions decide on the nature and level of their involvement in economic development activities in the context of 
a complex array of external and motivating factors. In addition, the literature suggests a strong correlation between 
level of economic development activity and change among selected academic policies. 

The study identified “motivating” factors responsible for encouraging increased institutional involvement among 
public institutions. Factors such as point of view of the president, of business leaders, and of state legislators and 
government; having a strategic plan; wanting to improve public relations and image; transmitting knowledge through 
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nontraditional teaching; increasing state appropriations; meeting public service obligations; generating new 
knowledge; and increasing corporate involvement appearto be the most influential. According to the study, the single 
most important key to successful institutional involvement in economic development is leadership. Entrepreneurial 
leadership by the institutional president is particularly important. 



Recommendations 

o Be specific in defining what “economic development” means for your institution in terms of fulfillment of your 

institutional mission. The goals of higher education institutional involvement in economic development 
activities must be clearly defined in terms of statewide economic goals, regional economic goals, and 
institutional goals. 

o Document your activities in this area for the purpose, among others, of fulfilling your mission and preparing 
for your next self-study. 

o Conduct an examination of internal institutional policies and procedures to identify barriers in meeting stated 
institutional, regional, and state economic development goals. Identify negative as well as positive costs 
associated with changes in internal policies and procedures. 

o Build relationships, working partnerships, and improved communication with other institutions, business and 
industry leaders, and state government regarding economic development needs and strategies. 



Conclusion 

Mission statements define the basic character of all institutions. Every public institution in Oklahoma has an economic 
development function. Oklahoma higher education plays a key role in the state’s economic development by 
customizing innovative programs and systems for business and industry. 

Over the last two years, the State Regents have funded more than $8 million in grants for 28 economic development 
projects throughout the state, attracting over $25 million in matching funds. These economic development grants as 
well as other economic development activities have given institutions the ability to link academic programs to 
economic outcomes. Economic development grants can foster institutional change both as an institution and within 
an academic program. 

The planning and implementation of this statewide initiative has strengthened Oklahoma’s economic structure while 
supporting the state and various institutions to work together to fulfill the mission of universities and colleges in 
promoting economic development. 



Carolyn A. Taylor is an Assistant Professor and NCA Coordinator at Rogers State University in Claremore, Oklahoma. 

Sid Hudson is Vice Chancellor of System Advancement and Economic Development for the Oklahoma State Regents 
for Higher Education in Oklahoma City. 
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Quality and Baccalaureate 
Curriculum Design 



Neala Schleuning 

Calls for change in higher education are loud and insistent: demands for a quicker response to train the nation’s 
workforce, for greater flexibility and efficiency in delivery methods, for the adaptation of new technologies to 
education. Increasing competition in the information delivery “industry” requires us to be more responsive, to do 
everything better, faster, easier, cheaper. In this headlong rush into the new information-based economy, we have 
not sufficiently explored the impact of these many changes on the integrity and quality of the baccalaureate 
curriculum-especially traditional design principles of consistency, coherence, and the progressive and recursive 
development of cognitive skills. 

Much of the contemporary literature on quality and accountability in higher education focuses on the application of 
industrial efficiency models in which the curriculum is subjected to the measurement of largely quantitative inputs, 
throughputs, or outputs. Concepts of intrinsic quality have been replaced by complex mechanistic models of quality- 
as-process-and-system. While this focus on the quantitative and measureable can provide valuable information for 
better managing resources, it ignores the more important subtle and qualitative curricular elements of structural 
integrity, thoughtful design and continuity, and cognitive integration across the curriculum. In our eagerness to 
measure ends, we have ignored the means that define a “quality” baccalaureate experience. 



The Baccalaureate Curriculum 

The baccalaureate curriculum has multiple objectives: it is designed to educate the whole person; to prepare him or 
her as a citizen and productive member of the community; to develop cognitive skills at an advanced level; and to 
initiate students into the practice of a particular body of knowledge-the academic discipline. Through careful design 
of the curriculum, the baccalaureate typically 

reaches beyond mere comprehension or acquisition of factual data or information to an awareness of its 
implications and meaning (breadth and depth). ... Baccalaureate recipients possess advanced communication 
and literacy skills. . . . They recognize and deal with complexity, ambiguity, and uncertainty. . . . They are skilled in 
using various methods of inquiry: scientific, philosophical, problem-solving and artistic. . . . They are able to analyze 
and evaluate both quantitative and qualitative data toobtain meaningful information. They can integrate knowledge 
obtained from different sources and disciplines. (Academic Senate of the California State University, 1997, 6-7) 

The primary objective of the baccalaureate is to develop higher-order cognitive skills; it is designed to teach students 
how to think, how to discern meaning. Baccalaureate curriculum design is based in large part on Benjamin Bloom’s 
taxonomy of learning development. Developed in the early part of the twentieth century by a large committee of 
educators that was chaired by Bloom, the theory has become the philosophical underpinning of most subsequent 
theories of learning and curriculum development. The taxonomy describes “levels" of learning and the orderly 
progression of learning from one level to another to achieve higher-order thinking skills. Its principles are applied at 
all educational levels and are so embedded in the curriculum development process that they are often unconscious. 

Because the baccalaureate degree is designed specifically to develop cognitive skills, Bloom’s formula plays a major 
role in shaping the design and determining the outcomes of a baccalaureate education. It assumes that intellectual 
skills can be taught through conscious curriculum design. This objective is accomplished by taking the student 
through an orderly, hierarchical, and recursive progression of coursework designed to enhance the cognitive skills. 
The idea of development, of progressing from one stage to another, one level of skill and ability to another, is at the 
heart of all cognitive learning theory. While Bloom’s formula does not give us the precision to measure degrees of 
cognitive complexity, it does provide a broad outline and a map of the learning process. 
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Acclimating students to the academic discipline is a primary means of developing higher-order cognitive skills. 
According to the Oxford English Dictionary, the discipline is “the act of teaching or instructing; that which is taught; 
instruction imparted to disciples or scholars." It is antithetical to the doctrine, or abstract theory, which is the property 
of the doctor or teacher. A discipline is both a subject area and the organized process of learning howto apply one’s 
mind and intellectual skills to that subject area and by analogy to other areas of knowledge. A discipline connotes 
something broader than a major in that it reflects a general approach rather than content. 

"What then is the practice of the academic disciplines? ... It is the practice of inquiry. . actively aimed at satisfying the 
human desire to know.”. . . Mastery of subject matter may be essential to the mastery of an academic discipline, but the 
discipline-the inquiry-is what leads to knowledge. Baccalaureate education is argument about interpretations. 

(Thomas Green, in Association of American Colleges, 1985, 28, emphasis mine] 



Elements of Design in the Baccalaureate Curriculum 

The baccalaureate curriculum was conceived as an integration of multiple bodies of knowledge, not merely the sum 
of individual courses taken. Broad areas of knowledge-general education, the major, and related minor areas-were 
to be integrated with one another in a meaningful way. A recent report by the Association of American Colleges and 
Universities noted that we have lost sight of this broader plan for the baccalaureate: “By allowing course credits to 
stand as a surrogate for learning, we have allowed ourselves to shirk the responsibility of developing a rigorous 
definition of what the baccalaureate degree should mean" (Schneider and Shoenberg, 1995, 19]. 

Traditionally, the baccalaureate curriculum is composed of several key components of design: integration of 
disciplines (particularly through a general education program], coherence at all levels of the degree, breadth and 
depth in knowledge areas, and levels of learning that reflect the progressive and recursive nature of knowledge. 
General education comprised one-third of the baccalaureate, and the major another third. The final third of the degree 
provided students some flexibility to study another discipline in a “minor,” or to explore elective areas of knowledge 
unrelated to the major or in afield that was either critical to or supportive of the major. The enmeshment of these three 
basic building blocks provided coherence to the baccalaureate curriculum, and the breakdown of this coherence is 
one of the major issues facing baccalaureate education today. 

□ Coherence 

The most important element in curriculum design is coherence. Related elements include the principle of 
sequencing On some fields, but not in others], the relationship between core courses and electives, the integration 
of knowledge across disciplines, and attention to epistemological questions. Michigan State University identified 
the following elements of coherence in the baccalaureate: 

Coherence-emphasizes either sequential or lateral design of interrelatedness of courses; integration- 
emphasizes interdisciplinary/multidisciplinary within courses and sets of courses , between and among 
Core studies and the more specialized major. . disciplines, then, become not just new bodies of information 
but new angles of vision; and synthesis-the frequent synthesis and reexamination of accumulated 
knowledge.. . . freshman-level understandings cannot remain unchallenged through the remaining years 
of an undergraduate education. ("Opportunities for Renewal,” 1988] 

□ Breadth and Depth 

An important objective of the baccalaureate is inculcating an appreciation of a variety of ways of approaching 
reality. This can be done by broad exposure to multiple disciplines as well as by a more intensely focused study 
of one area of knowledge, in a typical baccalaureate education, breadth is achieved at the lower division in general 
education and depth at the upper division in a specific discipline. 

Robert Zemsky has observed that the “commitment to breadth of study remains the fundamental characteristic 
of a liberal education" (1989, 16]. Breadth is often referred to as general education or liberal studies. It includes 
the traditional liberal arts and sciences such as literature, philosophy, history, the performing arts, humanities, the 
natural sciences, and the social sciences. Breadth is not just exposure to content, however. The primary objective 
of a general education curriculum is to enrich the epistemological tools of students, to teach them to think outside 
the context of their chosen field of study. It is based on the belief that the engineer will benefit from a grasp of 
aesthetics; the historian by studying human psychology. 
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A distinction should always be made between general education and introductory coursework to a discipline. 
General education courses are designed to give the non-major an overview of the tools of a discipline. Ideally, the 
general education curriculum should remain independent of the major area of study, and in many institutions 
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general education courses cannot be used to meet major requirements. Unfortunately, the uncontrolled growth 
of requirements for the major has accelerated pressures to integrate major requirements into the general 
education curriculum. Some institutions fence off significant portions of general education requirements and 
assign them to the domain of specific disciplines or broad cognitive areas. 

Why is breadth important? Exposure to the range of modes of apprehending and organizing knowledge is critical 
to the baccalaureate experience. At its very best, breadth should be crafted to provide students with the skills to 
integrate and interrelate multiple knowledge domains and multiple modes of inquiry. The cross-fertilization and 
integration of knowledge are especially critical to the creation of new knowledge. Recipients of the baccalaureate 
are liberally educated, not narrowly trained. A commitment to breadth is also an implicit recognition of the necessity 
fora rich and diverse curriculum. The importance of maintaining curricular diversity should not be underestimated: 
“humankind’s ultimate survival depends as much on a diverse pool of thoughts as on a diverse pool of genes” 
(Cowan, 1996, 8). 

In-depth study is generally undertaken in a specific discipline or major at the upper division level. One of the best 
definitions of depth in the curriculum can be found in the 1985 report, Integrity in the College Curriculum: 

Study in depth... is recognition of the degree of complexity and sophistication with which the various 
components are interrelated and understood.... A course of study has depth if it in fact offers a complex 
structure of knowledge. The comprehension of this structure-a decent understanding and control of it- 
is what we mean by study in depth.. . . Study in depth requires multiple dimensions; it cannot be reached 
merely by cumulative exposure to more and more of a specified subject matter. (Association of American 
Colleges, 1985, 28 and 29) 

The literature identifies a variety of indicators of depth: “significant concentrations within majors ... sufficient 
advanced coursework to ensure real depth ... [and the presence of] sequences of courses that lead students from 
the introductory level and through an intermediate state before culminating in one or more senior or capstone 
courses” (Zemsky, 1989, 6). A primary indicator of depth in the major is that “roughly one-third of the tagged 
courses in the domain had three or more prerequisites” (Zemsky, 1989, 19). Southeast Missouri State University’s 
definition of a major includes the following objectives: "Study in depth provides students with an understanding 
of the fundamental problems and arguments of a discipline or field of study, as well as their limits. It affords them 
practice with the tools of the subject, introduces them to its historical and philosophical foundations, and gives 
them a clear sense of its boundaries and its effectiveness as a means for understanding or serving human society” 
(Southeast Missouri State University, 1991, 19). 

□ Levels of Learning 

Consistent with Bloom’s taxonomy, colleges and universities have traditionally differentiated coursework into 
lower and upper divisions to reflect the intellectual developmental process. The expectation is that one cannot 
perform well at the upper division levels without the preparatory material and the cognitive skills gained at lower 
levels. Coursework at the lower division level is designed to provide the student with basic intellectual skills 
(reading, writing, and oral presentation), foundational coursework in the major, and exposure to the cognitive and 
epistemological structures of a wide array of academic disciplines. Because upper division coursework expects 
students to think at a higher level of analysis and in greater complexity, the tools of multiple disciplines and the 
skills necessary to apply them must be second nature to the student. 

Traditionally, lower division coursework had a variety of purposes: exposing the student to a breadth of knowledge 
across many disciplines, providing prerequisite courses for advanced study in a field, and/or introducing the 
student to the major. Upper division coursework addressed depth and coherence in the baccalaureate degree. In 
general, lower division coursework is introductory and requires no prerequisites. Lower division courses “increase 
the knowledge students have of subjects with which they are already familiar, introduce them to new subjects, and/ 
or establish a foundation for them to study a major subject in depth” (South Dakota Board of Regents). Individual 
courses often consciously emphasize breadth through the accumulation and comprehension of general knowl- 
edge in a variety of disciplines. Course titles often include the term introduction to or otherwise imply that the 
course is a survey or an overview of a discipline or field of study. 

In addition to exposing students to a range of knowledge, lower division courses also begin the process of training 
the mind. 

The primary function [of lower division courses]... is to begin making independent learners of students... . 

[Courses are] tightly structured, students receive considerable instructional guidance in the learning 
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process... instruction is normally information and emphasizes learning skills; entails the use of text 
materials or resources provided by the instructor Intellectual skills emphasized in lower division courses 
include comprehension, analysis, synthesis, evaluation, and application of knowledge, but these compe- 
tencies are not stressed to the same degree they are in upper division courses. [Southeast Missouri State 
University, 1991, 12-13, and South Dakota Board of Regents) 



Upper division courses build on the knowledge and skills learned at the lower division and are designed to develop 
students’ abilities to undertake advanced levels of learning consistent with Bloom’s taxonomy: comprehension, 
analysis, synthesis, evaluation, and application of knowledge. An increased engagement with the discipline atthe 
upper division level is designed to help students develop academic and intellectual sophistication. Upper division 
coursework should also comprise a significant percentage of degree requirements-between one-third and two- 
fifths of the degree. 

The student’s level of intellectual maturity and ability play an increasingly important role in upper division 
coursework as more of the responsibility for learning falls to the student. There is an assumption that students are 
more committed to the learning process. The primary function of upper division courses is “to refine students’ 
abilities as independent learners” (Southeast Missouri State University, 1 991 , and South Dakota Board of Regents). 

Definitions of upper division commonly include references to the ability to demonstrate “sophisticated understand- 
ing, and creative and imaginative synthesis” (AcademicSenate of the California State University, 1 997, 6-7), higher 
levels of abstraction (MHECB, 1 986), and the application of theory: “As a department we commonly talkof this level 
as the ‘theory courses.’ These courses are fairly specialized to allow more in depth analysis and should cultivate 
higher order thinking” (Speech Communications Department Position Paper, 1999). 

□ Designing the Major 

Policies vary, but generally there is an expectation that a significant portion of work at the upper division level will 
be undertaken in the major. The student undertakes an initial, systematic introduction to the work of the academic 
profession. Through immersion in the discipline, higher order thinking skills are further developed as the student 
is gradually exposed to more sophisticated research and analytical skills and is encouraged to apply those skills 
to current issues in the field and develop an appreciation for the creation of new knowledge. In general, heavy 
reading requirements to ensure immersion in the field often characterize upper division courses in the major. 

The program curriculum for the major should be designed to integrate lower and upper division coursework, and 
it should emphasize the development of advanced cognitive skills, including the following elements, to ensure its 
integrity and coherence: 

o A principle or principles of organization, and clarity of design 
o A progression of knowledge from introductory to final integrated project 
o A core of requirements common to all students 

o Limited electives, because "learning is cumulative across courses” and the learning experience is 
“structured to take time.... This recursiveness in the curriculum is the enemy of naive acceptance” 
(Challenge, 8-11). 

According to Michigan State University faculty, the major should 

teach students the history and tradition of the field, its social and economic implications, and the ethical and 
moral issues to be confronted; should contribute to common understanding rather than to competing and 
overly-specialized ends; have a distinctive and legitimate intellectual content of its own; have the capacity to 
enlarge, rather than to narrow, the vision of the student; relate the values of liberal learning to the discipline 
or profession; have both requisite complexity and relevance to later life; foster the student’s quest for informed 
judgment and the capacities for both appreciate and criticism; teach students to make interpretive decisions 
and to recognize the consequences of decisions and actions. ("Opportunities for Renewal") 



□ The Faculty 



0 
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Quality of instruction, the abilities of individual faculty members, and continuity of learning with a particular group 
of scholars are also critical elements of program design. This design element is based on the principle that it is 
the discipline’s right to credential the members of its profession. The academic credential guarantees the quality 
of instruction and assures the mastery of the discipline by the student and, by default, the ability of the individual 
to pass on the tools of the trade. 
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A related, but nearly forgotten, element of curriculum coherence is the idea that the credential is a function of the 
collective wisdom and philosophical bent of a particular group of faculty members at a particular institution. A 
major is not just the sum of its parts or an accumulation of courses from ten different programs. This element will 
be increasingly relevant in an era of distance learning at multiple institutions. Who will finally validate the student’s 
knowledge? 



Questions and Challenges 

o Should the baccalaureate be redesigned? Does it work as currently configured? Why? 

o The challenge of defining and describing design quality: What are the mechanisms for defining quality at the 
programmatic and degree level? 

o The challenge of being accountable for quality: How would you measure the outcomes of quality design? If 
you accept these guidelines, what is their relationship to student success? How do you know? 

o How do we maintain faculty control of the curriculum? How do we ensure continuity and coherence in the 
organization of knowledge? 
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Measuring Academic Quality 
in the Department of Defense 

(DoD) 



Leslye McDade-Morrison 



Why Is DoD Interested in Academic Quality? 

□ DoD Education and Training 

The DoD has one of the largest if not the largest, training system in the world. HON Rudy De Leon, as the former 
Under Secretary of Defense for Personnel and Readiness, stated in 1997 that the “Military Services and the DoD 
Agencies... annually spend $14 billion to provide 170,000 student years of education and training in over 20,000 
courses in schoolhouse training alone.” 

What are the schoolhouses that the HON Rudy De Leon referred to? 

□ Degree-Granting Institutions 

All of the degree-granting institutions are accredited by the appropriate regional accrediting body, as well as by 
specialized accreditors for engineering and medical programs. The institutions include, to name a few: 

o Three service academies (Westpoint, the Air Force Academy, and the Naval Academy) 
o A medical school 
o Graduate schools of engineering 
o An intelligence college 
o A language institute 
o Four war colleges. 

□ Military Education/Training Institutions 

The appropriate nationally-recognized or specialized accrediting body accredits some of these institutions. 

o 38 Army schools (Infantry, Armor, Air Defense Artillery, etc.) 

o 14 Navy/Marine Corps schools [Fleet Training Center, Trident Training Facility, etc.) 

o 1 7 Air Force schools that provide specific military job related training (Squadron Officer School, Air Force 

JAG School, etc.) 

□ Civilian Education/Training Institutions 

In addition, there are many institutions that provide job-specific training primarily to civilians, although there are 
some military students. These institutions provide specific training in: 
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o Polygraphy 
o Intelligence 
o Auditing 
o Acquisition 
o Resource management 
o Cryptology 
o Imagery and mapping 
o Equal opportunity 



Some of these institutions are accredited by the appropriate nationally recognized accrediting body. 

□ DRI Findings 

“DoD has many educational programs, but their quality is mixed. . . . faculties are often not challenged, and students 
are not inspired” (Defense Reform Initiative, 1997). This statement from the Defense Reform Initiative (DRI) is 
referring to the education programs for civilians, not for military personnel. The military’s education, training, and 
professional development system produces the finest military force in the world. Unfortunately, the civilian 
opportunities are not as well-defined as those for the military. Some see civilian education and training as less 
important than military education and training. For this reason, the DRI concluded that the DoD renders second- 
rate education, training, and professional development to its civilian employees. It is the goal of the DRI to raise 
the quality of civilian education, training, and professional development to “world-class” standards. 



What Is the DoD Doing About Academic Quality? 



□ DoD Chancellor for Education and Professional Development 

As a result of the findings from the DRI, the Secretary of Defense created a new organization, the DoD Chancellor 
for Education and Professional Development, with the mission “...to serve as the principal advocate for the 
academic quality and cost-effectiveness of DoD civilian education and professional development activities” 
(DoDD 5124.7, September 27, 1999). 

□ Metrics of Excellence Task Force 

The Secretary of Defense said that a regional or nationally recognized accrediting body approved by the 
Department of Education will accredit all institutions. To that end, we are in the process of assisting institutions 
to apply for and receive accreditation. Of course, the appropriate regional accrediting body already accredits the 
degree-granting institutions. 

On January 19, 2000, the Chancellor invited the heads of 25 institutions to the inaugural meeting of the Metrics 
of Excellence Task Force. These individuals became the Steering Group. Each member of the Steering Group 
appointed two individuals from their staff to participate in two separate working groups, Academic Quality Working 
Group fAQWG) and the Academic Resources Management Working Group (ARMWG). The AQWG metnbers were 
the senior academic officers (academic deans or provosts) of the institutions. The ARMWG members were the 
comptrollers, or resource managers, of the institutions. 

Each group met almost monthly and kept in very close touch via email. AQWG began by drafting academic 
standards in three categories, curricula, faculty/staff and students. 

□ What Were the Results? 

AQWG developed: 

o DoD academic quality standards for faculty, students, and curricula 
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o DoD academic quality levels to measure the degree to which an institution has attained the standards 
(based on “The Competency Standards Project: Another Approach to Accreditation Review," CHEA 2000). 
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ARMWG developed: 

a process to identify and report all costs related to the educational mission of the institution. 




□ Why Quality Levels? 

Quality levels enable us to look at the institution as a whole rather than as disparate measures of the twelve 
academic quality standards. For the four standards in the category of curricula, we included all the important 
aspects of those standards by creating five quality levels. Level 1 describes an institution of low quality, and Level 
5 describes an institution of high quality. The levels grow in complexity from Level 1 to Level 5 in two ways. First, 
there are more elements to satisfy at Level 5 than at Level 1 , and second, the elements in Level 5 are more complex. 
It is possible for an institution to satisfy all the elements of a level plus elements of some higher levels. 

To qualify at a particular level, the institution will have to satisfy all elements in a level. If an institution desires to 
move up a level, it knows exactly what it has to do to achieve the next higher level. Some may decide that it is too 
expensive to move to a higher level. However, this rating system may also provide the institution a needed 
requirement to ask for and receive more funding. 

□ Examples of Quality Levels 

Examples of some of the quality levels from the curricula standards are in the following three tables. Bold type 
identifies the differences among the levels. Each numbered item within a level is called an element. In this example, 
Levels 1 and 2 have two elements each, and Levels 3, 4, and 5 have three elements each. In the actual curricula 
quality levels, there are four elements in Level 1 and twelve elements in Level 5 (see Appendix). 

□ Measurement Scale 

The measurement scale between levels is identified by the words “few, if any," “few,” “some," “most,” and “almost 
all to all.” We defined them as follows: 




Few, if any: 
Few: 
Some: 
Most: 
Almost all to all: 



none or a very small number, usually in the range of 0<15°/o 
a small number, usually in the range of 15<30% 
a modest number, usually in the range of 30<70% 
a large number, usually in the range of 70<85% 

a very large number or approaching the entire population, usually in the range of 85 to 
100 % 



The use of numerical percentages standardizes the data from the institution. Without these percentages, one 
institution may determine that “few, if any” means 1 percent and another institution may conclude that it means 
10 percent. These differences in interpretation would make the data unreliable and eliminate the possibility of 
institutions benchmarking across categories. 
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□ Quality Levels: Element 1 

This first element asks the institution to: 

o Evaluate how many programs/courses assess student achievement of required learning outcomes 
(Levels 1 -5) 

o Determine the amount/type of evidence that the required outcomes were attained (Levels 1-5) 
o Include stakeholders in the assessment (Level 5). 



0 Chronicle of Higher Education. Like the higher education community, the DoD is very interested in 
measuring the degree to which a student has learned and is able to apply course concepts. James Rogers, 
the Executive Director of SACS, stated: “The regionals are all becoming less concerned with how colleges 
are structured and more focused on what kinds of graduates they produce” ( Chronicle of Higher Education, 
July 2000). 

0 Virginia Blue Ribbon Commission. “In light of the sharp competition for limited public resources from every 
sector, the citizens of Virginia cannot be expected to continue to support substantial increases in higher 
education spending unless they are presented with a body of meaningful evidence that demonstrates the high 
educational quality their support makes possible. To develop such evidence, we must first recognize that 
excellence in higher education is best evaluated by. ..not what they know when they enter, but by what they 
know when they graduate.. . .” (Final Report on the Governor's Blue Ribbon Commission on Higher Education t 
February 2000). 



low Quality | Academic Quality: Curricula (Element 1 ) 

• 



High Quality | 

► 



Level 1 Level 2 Level 3 Level 4 Level 5 



1. Few, if any, pro- 
grams/courses use 
assessments to cer- 
tify student achieve- 
ment of learning out- 
comes, and there is 
little or no evidence 
that students attain 
required outcomes. 



1. Few programs/ 
courses have assess- 
ments in place to cer- 
tify student achieve- 
ment of learning out- 
comes, and there is 
some evidence that 
that students attain 
required outcomes. 



1. Some programs/ 
courses use valid and 
reliable assessments 
to certify student 
achievement of learn- 
ing outcomes, and 
there is convincing 
evidence that stu- 
dents attain required 
outcomes 



1. Most programs/ 
courses use valid and 
reliable assessments 
to certify student 
achievement oflearn- 
ing outcomes, and 
there is convincing 
evidence that stu- 
dents attain required 
outcomes. 



1. Almost all pro- 
grams/courses use 
valid and reliable as- 
sessments to certify 
student achievement 
of learning outcomes. 
The assessment in- 
volves students, 
graduates, faculty, 
and stakeholders, and 
there is convincing 
evidence those stu- 
dents attain required 
outcomes. 



□ Quality Levels: Element 2 

In DoD, we used the term in appropriate programs/courses because not all institutions have programs or courses 
that require certifying, licensing, or national examinations. Therefore, an institution that does have this requirement 
will have to satisfy this element, those that do not have this requirement are exempt from satisfying this element 
(Levels 1-5). 




In addition to monitoring the percentage of students who attain the established target rate for certification, 
licensing, or exams, a high-quality institution analyzes the sub-sco res to identify deficiencies and then incorporate 
that information in a plan to improve the curriculum (Levels 4 and 5). 
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Low Quality 


1 Academic Quality: Curricula (Element 2) 


High Quality | 
















Level 1 


Level 2 


Level 3 


Level 4 


Level 5 


2. In appropriate pro- 
grams/courses, few 
if any students achieve 
established target suc- 
cess rates on certify- 
ing, licensing, and 
national examinations. 


2. In appropriate pro- 
grams/courses, few 
students achieve es- 
tablished target suc- 
cess rates on certify- 
ing, licensing, and na- 
tional examinations. 


2. In appropriate pro- 
grams/courses, some 
students achieve 
established target 
success rates on cer- 
tifying, licensing, and 
national examinations. 


2. In appropriate pro- 
grams/courses, most 
students achieve 
established target 
success rates on cer- 
tifying, licensing, and 
national examina- 
tions. Sub-scores are 
analyzed. 


2. In appropriate pro- 
grams/courses, all 
students achieve 
established target 
success rates on 
certifying, licensing, 
and national exami- 
nations. Sub-scores 
are analyzed and the 
results are used to 
improve the curricula. 



□ Quality Levels: Element 3 

In this era of increased emphasis on computer skills and online learning experiences, we felt that it was critically 
important for institutions to consider developing a plan to manage technology throughout the institution [Levels 3-5). 

The highest quality institution will review its technology plan annually to ensure that it still meets the needs of the 
institution, and its student population and continues to look toward the future [Level 5). 



Low Quality 

• 



Academic Quality: Curricula (Element 3) 



High Quality | 

► 



Level 1 


Level 2 


Level 3 


Level 4 


Level 5 






3. A technology plan 
exists for maintenance, 
support, and infra- 
structure upgrades. 


3. A technology plan 
exists and is reviewed 
regularly for mainte- 
nance, support, tech- 
nology, and infra- 
structure upgrades. 


3. A technology plan 
exists and is reviewed 
annually for mainte- 
nance, support, tech- 
nology, and infra- 
structure upgrades. 



□ Quality Level Characteristics 




o Outcomes-focused 
o Holistic view 

o Self-scoring and self-reporting 
o Not prescriptive 

o Flexible use-institutions, programs, traditional course delivery, distance learning 
o Uses existing data 
o Promotes consistency 
o Allows for benchmarking 
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Implementation 

The goal is to develop and implement a DoD online reporting system that captures data on academic quality and cost- 
effectiveness and generates reports to the institutions and other decision makers. 



Curricula Levels of Quality: An Illustration 

The graphic below visually illustrates the complexity of the quality levels. 



Academic Quality: Curricula 
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Tribal College Officials' Views and 
Perceptions Toward Accreditation 



Betsy Putman 



Introduction 

The purpose of this study was to assess how the views of tribal college administrators, faculty, and governing board 
members have changed toward accreditation since 1982. McDonald’s 1982 study 1 , “An Assessment of Accreditation 
Practices in Developing Indian Community Colleges Compared with Non-Indian Community Colleges in the 
Northwest” served as the comparison model. Surveys were administered to the faculty, staff, and board members at 
the same tribal colleges that participated in the 1982 study. The responses were compared to the 1982 results to see 
how their views toward accreditation have changed. 



Results 

1 . How do tribal college administrators rate the importance of various accreditation functions? 

o Fostering excellence 
o Encouraging institutional improvement 

o Appropriate educational objectives 
o Protecting against encroachments 

The importance of the four accreditation functions was highly rated. The importance of protecting institutions 
against encroachments was the least highly rated of the four functions. Fewer people answered this item. Judging 
from the comments, this was due to lack of understanding of the question [respondents were not clear as to what 
“encroachments” referred]. In general, it appeared tribal college officials believed the functions of accreditation 
are important. 

2. Is specialized accreditation or institutional accreditation more important to tribal colleges? 

Institutional accreditation was rated as more important than specialized accreditation, although a number of 
respondents said the two types of accreditation were equally important. This latter response may be dependent 
upon individual respondent disciplines because some programs such as nursing require specialized accreditation. 
Still, institutional accreditation was rated most important. A faculty member at a four-year college said, "Without 
institutional accreditation, it would often be impossible to get specialized accreditation.” 

3. What is the impact of accreditation on tribal colleges? 



0 

ERIC 



o Effects ot self-study 
o Accuracy of visiting team’s judgments 
o Help/hindrance of visiting team 
o Budget allocation changes 
o Helped desirable change 
o Hindered desirable change 
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o Prevented undesirable change 
o President recommended changes 
o Burden of accreditation 
o Costs/benefits of accreditation 

The answer to this question covered a wide range of issues. The importance of the institutional self-study effects 
was rated highly by respondents. Some respondents who rated the self-study impact as low commented that their 
colleges did not follow-up on the self-study findings. The most frequent response for the accuracy of the visiting 
team’s judgments was between “moderate” and “a great deal.” The help/hindrance of the last visiting team's 
judgments and suggestions was rated as “helpful” or “very helpful.” Each of the above three items had a large 
number of “no opinion” responses. This may be because many participants were relatively new to their college so 
they may not have been there when the last self-study and/or team visit occurred. 

Approximately half of the presidents made budget changes and most presidents recommended changes (in areas 
other than the budget] as a result of the self-study. Most respondents highly rated the extent to which accreditation 
helps desirable change. A faculty member at a four-year college commented, “Because of the ‘newness’ of most 
Tribal Colleges, I believe the accrediting process offers a very important standard for individuals and institutions 
new to education.” The extent to which accreditation hinders desirable change responses differed significantly 
between the two studies, and is discussed below. Most respondents believed that accreditation does not prevent 
undesirable change, and the next most frequent response was that accreditation had moderately helped prevent 
undesirable change. Only presidents responded to the question of whether institutional accreditation had been 
burdensome. Their responses were fairly evenly distributed among the five response categories (“none” to “a great 
deal”]. Most respondents felt that the benefits of institutional accreditation exceeded the costs. 

The responses to questions #1, #2, and #3 illustrate the importance and significant impact of institutional 
accreditation on tribal colleges. This may be because small and new institutions are more likely to be highly affected 
by accreditation. In addition, accreditation legitimizes tribal colleges within the higher education community, so 
accreditation’s functions were viewed as important. 

The extent to which institutional accreditation hinders desirable change in the respective institution had a 
significantly different response pattern between 1 982 and 2000. See the table below. More people in the 2000 study 
responded that accreditation hinders desirable change. This may be because now that these institutions are 
established, they are seeking changes such as adding programs and degrees, and may now experience a different 
relationship with accrediting agencies than when they were new. Newer institutions are more focused on earning 
initial accreditation for their existing programs while established institutions may be seeking accreditation agency 
approval for changes in programs, degrees, etc. and/or accreditation reaffirmation. Therefore, participants in the 
2000 study believed accreditation hinders desirable change to a greater extent than participants in the 1 982 study 
did. One administrative officer at a four-year college said, “the self-study and evaluation visit work okay. The 
between-time hoops to add new programs, degrees, certificates, locations, and distance learning changes are 
overly restrictive.” 



The Extent to which Accreditation Hinders Desirable Change 

Answered by Presidents, Administrators, and Faculty 







None 

O 


1 


Moderate 

2 


3 


Total 


Year of study 1 982 


Count 


56 


26 


16 


9 


107 




Expected Count 


39.2 


29.8 


22.9 


15.1 


107.0 


2000 


Count 


40 


47 


40 


28 


155 




Expected Count 


56.8 


43.2 


33.1 


21.9 


155.0 


Total 


Count 


96 


73 


56 


37 


262 




Expected Count 


96.0 


73.0 


56.0 


37.0 


262.0 



No opinion responses: n = 44 

O 
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4. Does accreditation accurately monitor nontraditional education? 

Respondents felt that accreditation was moderately accurate or accurate in monitoring nontraditional education. 
Accreditation agencies are sometimes accused of lacking innovation and the ability to monitor properly new trends 
in higher education. Tribal college officials rated this item lower than the other functions of accreditation, perhaps 
illustrating that accreditation agencies need to improve in the area of monitoring nontraditional education. 

5. What is the involvement of tribal college governing board members in the accreditation process? 

o Board received self-study 
o Board discussed accreditation 

Most board members received a copy of the self-study and discussed the self-study and/or the visiting team report. 
Board members are often highly respected commu nity leaders on their reservation and their involvement, or at least 
awareness of, the accreditation process was valued by college administrators to the point where they were 
informed and involved in discussions of the process. 

6. How acceptable are alternatives to private, voluntary accreditation? 

o Continue/discontinue private accreditation 
o Private vs. government accreditation 

o Importance of continuing accreditation if eligibility of funds eliminated 

Most respondents believed that private voluntary accreditation should continue. They also believed that private 
accreditation was more favorable than state or federal regulation. Tribal colleges currently are heavily reliant on 
the federal government for operational funds, and therefore tribal college officials may prefer to keep the 
accreditation process in the private sector so as to be less dependent on the federal government. Most respondents 
believed private accreditation should be continued even if the connection with eligibility of funds was eliminated. 
Tribal college officials believed the importance of accreditation extends beyond its link to eligibility for federal 
funds. For example, one person said private accreditation should be continued because it allows students to 
transfer courses to other institutions. 

7. How acceptable are changes to the accreditation process? 

o Professional evaluators on visiting teams 

o Non-educators on visiting teams 

The desirability of visiting teams composed of full-time professional evaluators or having non-educators serve on 
visiting teams provided a wide array of responses from very undesirable to very desirable. Some people thought 
it was favorable to involve full-time professional evaluators and non-educators, while others feared these people 
would not have the pro per educational background to make knowledgeable judgments.Tribal college officials may 
be more open to changing the composition of the visiting teams because they believed the current visiting team 
members do not adequately assess their institutions. One suggested change was to include more tribal college 
or other minority institutional representatives on visiting teams. 

8. Is accreditation meeting the tribal college needs? 

o Success of accreditation 
o Indian accreditation association 
o If so, replace existing associations 

Many of the participants responded “don’t know” to the question, "Does accreditation work?” This may illustrate 
that this broad question was difficult to answer. Of those who chose "yes” or “no,” the vast majority of people 
responded that accreditation works. Respondents were evenly divided on the question of whether an Indian 
accreditation agency should be formed. As described below, the different responses were received from American 
Indians as compared with Anglos. Twice as many people believed that an Indian accreditation agency should not 
replace the current accreditation agencies. Comments showed that people believed an Indian accrediting 
association should complement the existing agencies to aid with the cultural understanding of tribal colleges. 
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Response Patterns by Ethnicity 

Further analysis examined different response patterns of American Indians and Anglos. (The mixed and other 
categories were too small to analyze.) This comparison was for the 2000 study only as demographic data were not 
available for the 1982 study. Three items produced significant differences. 

0 More American Indians believed accreditation’s function of protecting institutions against encroachments 
was important. This may be explained by the relationship between tribal councils and tribal colleges. This 
complex relationship, and how it is affected by accreditation, is the topic of Wabaunsee’s [1 998) dissertation 
“Accreditation, Tribal Governments, and the Development of Governing Boards at Tribal Colleges in Montana 
and Washington.” It appears American Indians believed accreditation should protect the colleges from tribal 
council encroachments. Several respondent comments addressed this issue. A faculty member at a two-year 
college said, “Tribal politics, if left unchecked without buffers/barriers, can jeopardize our basic educational 
foundations. Controls need to be in place for external as well as internal interference that can thwart the 
educational process.” 

0 American Indians rated the importance of institutional self-study effects higher than Anglos. The cause for 
this difference was difficult to determine. 

0 Significantly more American Indians than Anglos believed an Indian accreditation agency should be formed. 
Many respondents commented on this item. Some American Indians believed that accreditation agencies do 
not properly account for cultural differences on which the colleges are based. Other respondents believed that 
there should not be an Indian accrediting association, as it might not be regarded as an equal to the existing 
accrediting agencies. 



Recommendations 

The following recommendations are based on the findings from this study. 

0 Tribal colleges should continue to obtain and maintain accreditation with the regional accrediting 
associations. As the impact and the importance of institutional accreditation were highly rated by 
participants, accreditation is viewed as very important to the success of the tribal colleges. 

0 Tribal college representatives and regional accrediting association representatives should con- 
tinue working together to help the latter understand the unique characteristics of tribal colleges. 

Many tribal college officials feel that the missions of their colleges are not adequately understood by 
accreditation officials. Tribal college officials and accreditation agency officials should collaborate so that the 
mission of each is adequately understood by the other party. This may help the accreditation process proceed 
more smoothly. In addition, more tribal college representatives should serve as accreditation Consultant- 
Evaluators. This may help alleviate the issue of visiting team members not fully understanding tribal college 
missions. 

0 Tribal colleges and AIHEC should carefully investigate the benefits and costs of establishing their 
own accreditation association. Many tribal college officials, especially American Indians, favor the 
establishment of an Indian accreditation agency. A feasibility study should be undertaken to see if the benefit 
to the tribal colleges would outweigh the costs of establishing this organization. This feasibility study should 
include a mission statement so that the purpose of and need for the organization are clear. This statement 
should also clarify the Indian accreditation agency’s relationship to existing accrediting agencies. 



Areas for Further Research 

The areas for further research are based on the recommendations and the conclusions above. The first two areas for 
further research are suggested changes to a future tribal college accreditation study. The next area focuses on high 
turnover of tribal college faculty and staff. The following areas focus on further research on the impact of accreditation 
and potential changes in the accreditation process. 

A future accreditation study should include all tribal colleges. The 1 4 tribal colleges in this study were 
among the more established of the 33 tribal colleges. Do tribal college officials at newer institutions hold the 
same views toward accreditation as those at established institutions? Seemingly, newer institutions may have 
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different needs regarding accreditation Qnitial accreditation versus re-accreditation). It would be interesting 
to see how this affects views toward accreditation. Including all of the colleges could also provide a means 
of comparison between accredited and non-accredited institutions. 

0 For a future accreditation study, consideration should be given to revamping the study question- 
naires so more people are asked each item. This would present a more accurate picture of how the 
different groups view accreditation. For example, in the current questionnaire only administrators were asked 
to rate the importance of continuing or discontinuing private voluntary accreditation. General questions such 
as this could be asked of all participants to gain a better understanding of how tribal college officials view 
accreditation. 

0 The causes for high turnover of faculty and staff at tribal colleges should be studied. Tribal colleges 
sometimes have a difficult time retaining staff and especially faculty. Discovering the causes of this may lead 
to higher retention rates. For example, if Anglo employees are leaving due to cultural differences, cultural 
education programs could be established to address this issue. 

0 Factors that lead tribal colleges to successful initial accreditation should be studied. Accreditation 
clearly is very important to tribal college officials. The initial accreditation process can be overwhelming, 
especially to a minimally staffed, newly founded institution. Identifying factors that lead to successful initial 
accreditation may help newer colleges successfully navigate this process. This in turn will help these 
institutions become successful as accreditation enables funding, transfer of credits, etc. 

0 Ways to make accreditation less burdensome to tribal colleges should be analyzed. Although 
accreditation may be especially beneficial to new and small institutions, it may also disproportionately burden 
them as they have fewer resources (monetary, personnel, etc.) with which to carry out the process. Therefore, 
making accreditation less burdensome to these institutions could decrease the time it takes tribal colleges 
to achieve initial accreditation. 

0 The specific ways in which accreditation hinders desirable change should be studied. Significantly 
more respondents in the 2000 study believed accreditation hinders desirable change than in the 1982 study. 
Identifying the specific ways in which accreditation hinders desirable change could help the accreditation 
agencies strengthen their relationship with tribal colleges. 

0 Involving full-time professional evaluators and non-educators in accreditation visiting teams 
should be explored. Many respondents were in favor of changing the composition of visiting teams by 
including professional evaluators and non-educators. Research could determine the positives and negatives 
of these changes. Further research could also determine why these changes are desired by some tribal college 
officials. 

0 The reasons why American Indians would like their own accreditation agency should be studied 
further. Research could identify what needs are not being met by the current system and/or what is 
inadequate about the current system. Researching the causes for dissatisfaction with the current system could 
allow accreditation agencies to make changes to meet tribal college officials’ accreditation expectations. 



Summary 

Despite the majority of American Indians at tribal colleges wishing to establish an Indian accreditation agency, tribal 
college officials are generally satisfied with accreditation. Most aspects of accreditation were highly rated, with a few 
exceptions. Respondents in this current study believed that accreditation hindered desirable change more than 
respondents did in 1982. Respondents for both studies did not rate the accuracy of accreditation in monitoring 
nontraditional education as highly as other accreditation functions. In addition, respondents for both studies did not 
believe accreditation prevents undesirable change. Other than the above exceptions, the impact and functions of 
accreditation were highly rated. Most respondents believed private, voluntary accreditation should continue. 




The responses for the three items that varied by ethnicity of the participants (the importance of accreditation in 
protecting institutions against encroachments, the effects of the self-study, and the desirability of the establishment 
of an Indian accreditation association) seemed to be primarily attributable to cultural issues. Some tribal college 
officials, especially American Indians, believed accreditation associations do not sufficiently account for the cultural 
components of tribal colleges. 
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Although dramatic changes in accreditation and in the tribal colleges have occurred since 1982, attitudes and 
perceptions toward accreditation remain largely unchanged. However, the issue of accreditation warrants further 
exploration as accreditation continues to be a prominent issue in tribal colleges. 



Reference 

1. The study was completed in 1982, but the questionnaires were administered in Fall 1980. 



Betsy Putman, Associate Director of Development, Northern Arizona University ; Flagstaff, Arizona. 



Participating Colleges 



School 


Fortner Name/ 
Year Changed 


Location 


Accrediting 

Body 


Blackfeet CC* 




Browning, MT 


Northwest 


Cankdeska Cikana CC 


aka Little Hoop CC 


Fort Totten, ND 


North Central 


Din6 College 


Navajo CC/1997 


Tsaile, A Z 


North Central 


Dull Knife Memorial C 




Lame Deer, MT 


Northwest 


Fort Berthold CC 




New Town, ND 


North Central 


Little Big Horn C 




Crow Agency, MT 


Northwest 


Nebraska Indian CC 




Niobrara, NE 


North Central 


Northwest Indian C 


Lummi CC/1989 


Bellingham, WA 


Northwest 


Oglala Lakota C 


Oglala Sioux CC/1983 


Kyle, SD 


North Central 


Salish Kootenai C 




Pablo, MT 


Northwest 


Si Tanka C 


Cheyenne River CC/1999 


Eagle Butte, SD 


North Central 


Sinte Gleska University 


Sinte Gleska C/1992 


Rosebud, SD 


North Central 


Sitting Bull C 


Standing Rock C/1996 


Fort Yates, ND 


North Central 


Turtle Mountain C 




Belcourt, ND 


North Central 



*CC = Community College C = College 




*} , 
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The Utility of 
Learning Style Data for 
Learning Outcomes Assessment 



David Lopatto 
Carol Trosset 

The development of assessment practices to measure student learning outcomes on our campus originally took the 
form of devising tests, portfolios, capstone seminars, and other outcome measures. For most of our faculty, assessing 
student learning in general and in the academic major absorbs most of the time and energy they are able to devote 
to the issue. Both of the authors, however, have served as assessment coordinators to the college, and our experience 
has given us time to reflect on the overall approach to assessment. In what follows, we attempt to show that an 
exploratory study of student-centered assessment is possible and practical at most institutions. The key features of 
such a study are (1) an institutional research officer or assessment coordinator who is given time to conduct such 
studies, (2) easy-to-use instruments that require no special skills for scoring, and (3) the use of archival data that the 
institution may already possess. 

We observe that assessment plans at many institutions lack a procedure for assessing the baseline abilities, learning 
styles, and academic goals of students before the academic program is encountered. We believe that this lack may 
have been a result of the North Central Association’s emphasizing clear statements “regarding the institution’s 
expectations for student learning," but not clear statements regarding the student’s expectations for learning. 
Collecting information about students as they enter an institution or begin a course of study may complement an 
assessment program in several ways. First, it provides baseline information against which outcomes assessment may 
be compared. Second, it provides information about student goal setting (as in the case of intended major or career) 
that permits faculty to contemplate the congruency between student goals and program goals. Third, it indicates how 
well the student is “learning howto learn,” i.e., bringing appropriate learning strategies to academic tasks. Fourth, 
it provides evidence of differences in behavioramong individuals and groups that may aid planning foran increasingly 
diverse student body. Discussions of pedagogy and student diversity often make the assumption that different 
students arrive on campus with different “learning styles,” and that these learning styles should be accommodated 
in the teaching and assessment process. 

To engage in an exploration of these issues, we intentionally made use of measures of student behavior that were 
either already employed by the institution or were easy to use. The college application process yields archives of 
subject variables such as gender and ethnicity. In addition, the college routinely administers the CIRPfreshman survey 
provided by the Higher Education Research Institute. The CIRP survey, administered about one week before the first- 
year student begins classes, asks for a wide range of information including intended major, intended career, and self- 
ratings of student efficacy on a variety of behaviors. Our institution routinely uses the data to benchmark overall 
performance against similar institutions. For this study, however, we utilized data from those respondents who 
volunteered identification numbers that permit linking their responses with other information. This other information 
includes choice of major and overall grade point average. Finally, we used a brief learning style survey to assess self- 
reported learning style. 




Background 

Learning styles are patterns of preference or performance in learning. They have been related to student learning and 
motivation in achievement tasks. One theory of learning styles (Kolb, 1984) suggests that student learning can be 
assessed along two dimensions, that of concrete experience versus abstract conceptualization, and that of reflective 
observation versus active experimentation. Students with a distinct learning style may choose disciplines congruent 
with their style, perform best with pedagogy suited for their style, and benefit by interacting with students who 



8,9 



84/A Collection of Papers on Self-Study and Institutional Improvement, 2001 



demonstrate a different style. Learning style is often difficult or time-consuming to assess. Romero et al. [1992], 
however, introduced a brief, practical survey that yields information on student self-descriptions of the two 
dimensions. Students preferring concrete experience enjoy experiencing events, while students preferring abstract 
conceptualization enjoy developing concepts or theories. Students preferring reflective observation enjoy analyzing 
their experience, while students preferring active experimentation enjoy looking for the application of theories in 
solving problems. 

Both student goals and student learning styles influence student performance in higher education CSchunk, 1996). 
Goals motivate student behavior and may help a student focus on tasks relevant to learning. Most teachers know of 
students who came to a learning situation with clear academic and career goals. These students are often highly 
motivated, accept challenging work, and perform well. They are frequently able to compare their current performance 
to the criterion performance needed to reach their goals. Assessing student academic and career goals on an 
individual basis, then, may yield information that enhances teaching and learning. 

Estimates of self-efficacy on achievement tasks have also been shown to affect behavior. Schunk (1 996) defines self- 
efficacy as the judgment of one’s capabilities to organize and implement actions necessary to attain designated levels 
of performance. Self-efficacy is approximately a measure of self-confidence, but research has shown that it is formed 
by experience (Bandura, 1982). Assessing student self-efficacy estimates, then, may yield information that predicts 
student learning and student choices such as academic major. 



Sources of Data 

Our college routinely administers the Cooperative Institutional Research Program (CIRP) freshman survey, published by 
the Higher Education Research Institute. This survey is administered to new students during an orientation period a few 
days prior to the opening of the fall semester. A student’s CIRP data may be anonymous, but some students volunteer 
their Social Security numbers. It is thus possible to compare their responses to other sources of data. Other archival 
information, including declared major and overall grade point average, were obtained from the registrar’s office. 

The learning style survey was created and tested by Romero etal. (1 992). It consists of 14 questions. Respondents are asked 
to circle a number (1 to 6) that corresponds to their view of themselves. Pairs of statements, such as “I would describe myself 
as reflective” versus ”1 would describe myself as action oriented” are employed to derive scores on the two learning style 
dimensions, concrete-abstract (ACCE) and reflective-active (AERO). Scores may range from 7 to 42 on each scale. Romero, 
following Kolb (1 984), situates scores by treating the two learning style dimensions as perpendicular axes crossing at the 
means. This crossing results in four “quadrants” that representdifferent learning styles. Romero provided data to show that 
scores in the four resulting quadrants correlated with undergraduate major (see Table 1). 



Table 1. Description of the Four Learning Style Quadrants 



Quadrant 


Description 


Survey Scores 


Typical Majors 


Divergers 


Concrete, Reflective 


Low ACCE, Low AERO 


Languages, Theatre 


Assimilators 


Abstract, Reflective 


High ACCE, Low AERO 


Chemistry, Physics 


Convergers 


Abstract, Active 


High ACCE, High AERO 


Engineering, Computing 


Accommodators 


Concrete, Active 


Low ACCE, High AERO 


Business 



Subjects and Analysis 

A group of 1 98 undergraduates taking courses in either introductory statistics or introductory psychology were given 
the learning style survey by the first author, who was a course instructor. Of these, a search of CIRP archives yielded 
70 students who had provided identification on the CIRP. This consisted of 42 women and 28 men. 



Learning Style 

The group of 198 students provided average scores of 2 1.5 on the concrete-abstract scale and 24.9 on the reflective- 
active scale. There were no differences among ethnic groups or course groups on either dimension. There was a small 
but significant difference between men and women, with men scoring more toward the abstract side (mean = 22.6) 
of the concrete-abstract dimension than women (mean = 20.5). 

ERJC . 
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Learning style scores did not predict either grades in the course the student was taking or the overall grade point 
average of the student. Similarly, learning style alone did not predict the academic major the studentdeclared. In order 
to relate the various information from the CIRP with learning style data, we performed a cluster analysis procedure 
using SPSS for the 70 students for whom data were available. The analysis yielded five clusters of student profiles. 
One of these clusters included four students whose greatest common feature was a low grade in their psychology 
course. The low grade variable was not an input but an outcome of the cluster analysis. The other four clusters bore 
some resemblance to the four quadrants of learning style, including average learning style scores that placed the four 
clusters within the four quadrants [see Table 2). 



Table 2. Results of a Cluster Analysis in which Similar Student Profiles 
are Clustered Together On CIRP and Learning Style Data 



Cluster 


CIRP and Grade Characteristics 


Learning Style Means 


Overall GPA 


1 Cn = 21) 


Low efficacy in competitiveness 
and leadership. 


[19,19) Divergers 


3.09 


/ — s 

o 

II 

CM 


High efficacy in academic skills. 

All intend to go on to advanced degrees. 


[27,23) Assimilators 


3.47 


3[n = 12) 


High efficacy in math skills and 
in competitiveness. 


(30,31) Convergers 


3.19 


/ — s 

o 

CM 

II 

Cl 


High efficacy in verbal skills. 
No science majors. 


(19,26) Accommodators 


3.29 


5 [n = 4) 


High efficacy in social skills. 
Weak academically. 


(14,33) Accommodators 


2.40 



Closing the Feedback Loop 

We intend to present our findings at faculty forums at our institution. In particular, we plan to present the findings to the 
faculty who are preparing for next fall’s freshman seminars. These faculty teach groups of approximately 12new students 
in seminars that emphasize writing and discussion skills. Each faculty member becomes the academic adviser to the 
students he or she teaches. We believe that faculty who are familiar with the learning style quadrants may be able to 
diagnose reasons for student academic failure, both in these seminars and in other courses the students are taking. That 
is, advisers may be able to suggest that students adopt a style of learning that may be congruent with the pedagogy of 
the course. Measuring learning style reliably may be the first step to teaching optimal styles for learning. 



Conclusion 

Our exploration of learning style and other personal variables suggests that these “soft assessment variables may 
be useful in understanding the student’s perspective in the learning process. Estimates of self-efficacy and learning 
style may make modest but significant contributions to learning outcomes. 
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Assessment of Student Competence 
in Accredited Disciplines 



Catherine A. Palomba 
Trudy W. Santa 

During the past fifteen years, pressures for assessment of student competence have come from several groups, 
including students and their parents, employers of graduates, state legislators, and the general public. In addition to 
these stakeholders, regional and disciplinary accreditors have played particularly important and mutually reinforcing 
roles in encouraging assessment of student learning. Regional accreditors expect their member institutions to 
demonstrate that graduates are competent within their fields of study, thereby fostering assessment within 
disciplines. Similarly, the assessment expertise faculty develop as they respond to disciplinary accreditors helps them 
meet the assessment requirements of regional accreditors. 

We have observed that faculty in accredited fields have several advantages in conducting assessment. In addition to 
the impetus for assessment provided by disciplinary accrediting bodies, other factors include close faculty ties with 
alumni and employers, coherent programs of study, and opportunities to provide active learning experiences for 
students. Because we believe that faculty in accredited fields are assessment pioneers who have valuable experiences 
to share with their colleagues in other fields, we initiated a study of assessment in the disciplines of teacher education, 
pharmacy, nursing, social work, business, computer science, engineering, and the visual arts. Based on recommen- 
dations from knowledgeable individuals in the central offices of the relevant disciplinary accrediting bodies, we 
identified campus leaders to share the story of assessment in their disciplines. We asked each contributor to describe 
the current practice of assessment in that field on various campuses, to pay close attention to the role that specialized 
accreditors play in fostering assessment of student competence in the discipline, and to identify assessment lessons 
for faculty in other fields. The contributions from our authors demonstrate that, although each discipline’s assessment 
story is clearly unique, enough similarities exist to allow for meaningful and productive sharing of assessment 
experiences (Palomba and Banta, in press]. 



The Practice of Assessment in Selected Accredited Disciplines 

On many campuses, faculty in applied fields have made significant progress in carrying out assessment of student 
achievement in the major. They have reached consensus about various competences expected of students in their 
programs, developed and implemented a rich variety of assessment techniques, involved faculty and other 
stakeholders, and used assessment results to improve student learning. 

In each of the accredited disciplines examined for this study, faculty assess general education skills as well as 
specialized skills. Although the specifics differ within and across disciplines, faculty expect their graduates to develop 
the competences required by employers, including the abilities to think critically and solve problems. Students are 
also expected to develop appropriate values and attitudes. For example, both business graduates from Winthrop 
University and computer science graduates from Brigham Young University are expected to be competent in ethical 
reasoning, communication, and lifelong learning skills. In several cases, faculty have articulated expectations for 
competence that allow their students to build their skills over time. Teacher education faculty at Asbury College have 
created a Continuous Improvement Model to assess their students, and at King’s College the curriculum is structured 
around competence growth plans. 

In the disciplines we examined, faculty are most likely to use locally developed approaches. Often they draw on 
assessment instruments and methods that are already in place, including classroom assignments from capstone or 
other courses. Across the disciplines, faculty typically use multiple measures, including both quantitative and 
qualitative techniques. For example, on some campuses, nursing faculty use narratives to evaluate the ability of 
students to identify problems. As one might expect, faculty in applied fields use performance assessment extensively. 
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Similarto their colleagues elsewhere, faculty in the College of Design, Architecture, Art, and Planning at the University 
of Cincinnati use juried critiques and oral defense of theses to evaluate cognitive skills. Engineering students at 
Southern Illinois University Edwardsville (SIUE) complete major design projects that are presented both orally and 
in writing. Although less comfortable assessing attitudes and values, faculty in applied fields attempt to do so. For 
example, pharmacy faculty on various campuses use attitude surveys, preceptor evaluations, and the California 
Critical Thinking Dispositions Inventory for this purpose. 

Assessment leaders in the accredited disciplines featured in this study use various methods to help involve faculty 
in assessment. These include developmental workshops, faculty retreats, and attendance at assessment conferences. 
When facu Ity from the Division of Pharmacy Practice at the St. Louis College of Pharmacy were developing assessment 
criteria and rubrics in seven ability areas, each individual served on one of seven ability subcommittees. At the time 
they were preparing for a reaccreditation visit from their disciplinary accreditor, faculty at Virginia Commonwealth 
University attended biweekly meetings and participated in content teaching groups. 

Many faculty in applied disciplines have established close collaboration with external stakeholders, including alumni, 
employers, internship supervisors, and field instructors. For example, human resource management faculty at Ohio 
University have benefited greatly from the expertise provided by their advisory board of practitioners. 

Faculty on several campuses have organized their assessment processes in meaningful ways that facilitate the use 
of assessment results. Business faculty at Ball State University rely on undergraduate and graduate assessment 
committees to foster assessment activities. At Columbia College, the assessment program director for social work 
prepares an annual report that is the basis for recommendations by the social work advisory committee. On some 
campuses, the use of assessment information has led to significant change. For example, the College of Engineering 
at Virginia Polytechnic Institute and State University extended a writing program that was initially available in only 
one department to all departments. 



The Effects of Specialized Accreditation on Assessment in Selected Disciplines 

Since 1988 the U.S. Department of Education has required accrediting bodies to collect information about student 
learning as part of the accreditation process. Both the Council for Higher Education Accreditation [CHEA] and the 
Association of Specialized and Professional Accreditors [ASPA] expect accrediting organizations to emphasize 
assessment of student learning in accreditation reviews. In several disciplines including pharmacy, nursing, computer 
science, and teacher education, significant changes in expectations for graduates have also been a factor in 
encouraging specialized accreditors to expect assessment. To facilitate assessment, specialized accreditors provide 
appropriate resources, articulate expected competences for graduates, encourage the use of performance assess- 
ment, and expect their members to use assessment results to improve student learning. 

Specialized accreditors and professional associations use web sites, written materials, and conference sessions to 
support the assessment efforts of their members. AACSB— The International Association for Management Education 
has held three conferences devoted exclusively to assessment. In the early 1 990s the National Association of Schools 
of Art and Design (NASAD) began using sessions at its annual conference to encourage members to consider how 
assessment might be applied to art and design programs. The Commission on Accreditation [COA] of the Council 
of Social Work has introduced a faculty development program to assist faculty in creating assessment plans. 

Specialized accreditors typically expect their members to address both general education and discipline-specific 
skills in their assessment programs. The Council on Accreditation for social work expects baccalaureate and masters 
graduates to understand the positive value of diversity and to think critically. The Accrediting Board for Engineering 
and Technology [ABET) requires engineering faculty to assess the ability of graduates to function on multi- 
disciplinary teams and to understand ethical responsibility. Rather than developing exact statements that its members 
must adopt, in most cases an accrediting body expects its members to develop campus-specific statements to address 
the outcomes the accreditor has endorsed. 

Several specialized accreditors have encouraged the use of performance assessment. In 1990 the Council of Arts 
Accrediting Associations urged its members to focus assessment efforts on the ability of students to “integrate 
knowledge and skills comprehensively on professional work" [p. 6). Accreditors in pharmacy and teacher education 
have also encouraged performance assessment. 



0 
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Specialized accreditors have recognized the value of including stakeholders in the assessment process. The social 
work COA encourages its members to collect evidence from employers of graduates, field instructors, and clients. 
Typically, ABET visiting teams include a number of individuals from industry as well as academe. 
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Specialized accreditors expect their members to use assessment results to improve student learning. The Commission 
for Collegiate Nursing Education, a new accrediting body established in 1997, encourages assessment as a 
“conversation” aboutthe knowledge and skills nurses will need in the future. In some disciplines, the current emphasis 
on institutional mission and goals has provided more flexibility in the way faculty prepare for accreditation. For 
example, AACSB no longer requires its members to submit specific templates or charts, although specific types of 
information must be provided. 



Lessons and Challenges Across Accredited Disciplines 

The experiences of faculty in accredited disciplines illustrate several lessons that are common to successful 
assessment, including the need for patience, flexibility, faculty involvement, administrative leadership, and commu- 
nication. One of the most important lessons demonstrated by this study is the need for assessment to support the 
learning process that has been adopted in the discipline. In both teacher education and pharmacy, assessment 
innovations have been part of a larger set of changes that affect the way students are taught and evaluated. 
Educational leaders in pharmacy have recommended the adoption of an ability-based curriculum in order to support 
a shift in mission toward providing patient care rather than products. To implement this curriculum, faculty are 
developing assessment methods that provide feedback to students so they can monitor their own progress. 

In several disciplines, faculty have struggled to create successful assessment programs. Faculty in both nursing and 
engineering have sometimes found the task to be difficult. Those who have been successful have needed to be flexible 
in their approaches and have often modified their assessment techniques. For example, business faculty at SIUE 
added a letter grade to their senior seminar in order to increase student motivation to do well in the course. 
Successful assessment programs are those that involve faculty in a variety of roles and seek approval of the faculty 
as a whole atseveral points in the assessment process. For example, atSeton Hall the work of small groups of business 
faculty were eventually presented to the entire faculty for endorsement. 

On several campuses, assessment has changed the faculty culture. But this does not happen quickly. Nursing faculty 
at Indiana University spent several years developing and revising their curriculum and assessment processes. Faculty 
culture in the College of Business at Northern Arizona University began to change only when faculty began to think 
of the business core as a joint venture. 

Specialized accreditors need to do their part to ensure that assessment is successful. Visiting teams must be well- 
prepared to evaluate assessment activities on college campuses. David Dill points out that “good reviewers are gems,” 
but reviewers can also be unprepared or unrealistic [1998, p. 21). To be of most value, specialized accreditors must 
concentrate their efforts on helping faculty improve their programs. 

In spring 2001 , the results of this study will appear in a Stylus publication entitled Assessing Student Competence in 
Accredited Disciplines: Pioneering Approaches to Assessment in Higher Education. In addition to thirteen contributing 
authors, the leaders of the disciplinary accreditation organizations featured in the study assisted us in creating this 
volume. They expressed support for our project, helped us identify assessment experts in the disciplines, and provided 
assessment-related materials. The web sites of CHEA, ASPA, and several disciplinary accreditors also provided 
valuable information. 
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Toward Consistency of Assessment 
in Writing Seminars 



Dennis Temple 
Louise Love 

Roosevelt University is private institution serving some 7000 students at two campuses, one in downtown Chicago 
and the other in Schaumburg, a northwest suburb. The only universal requirement for undergraduates is two 
semesters of composition. Otherwise, each of the five colleges in the university controls its own general education 
system. 

The College of Arts and Sciences, the largest and most diverse of the five colleges, has a three-tier general education 
program. Students who enter the college with freshman standing are required to take a two-semester great books 
course. Sometime during their sophomore or junior year students must take a one-semester signature course on an 
aspect of urban-suburban life. Finally, students who are near graduation must take a senior seminar, the college’s 
capstone course. Signature courses and senior seminars are developed by faculty around their own interests under 
the supervision of the General Education Committee. Students can choose a signature course or senior seminar from 
an array of different offerings, around five to seven at each campus. 

The senior seminar was created as a course that would serve the needs of students who were about to leave the 
university. The faculty committee that designed the course decided to require that senior seminars be interdiscipli- 
nary. The point of this was to give students a more realistic model for problem solving, since real problems seldom 
if ever fall neatly into disciplinary boundaries. The seminar was also designed to give students a final opportunity to 
sharpen research and writing skills. All senior seminars, regardless of content, require students to do a major research 
paper of fifteen to twenty pages. 



Assessment Comes to Roosevelt 

In the fall of 1995, in preparation for its NCAsite visit, Roosevelt was required to submit a comprehensive assessment 
plan. The plan that we submitted called for the hiring of four Assessment Coordinators and the establishment of a 
University-wide Assessment Committee. The committee was formally constituted in the fall of 1 996, after the site visit. 
One of the authors of this paper was named co-chair of the Assessment Committee and Coordinator for General 
Education Assessment. 

In retrospect, it is fair to say that we were naive about the whole process of assessment. Some of us thought that 
assessment would be a relatively straightforward process of implementing our assessment plan. Many more thought 
that assessment would reveal little that we didn’t already know. After all, hadn’t we been assessing (grading) students 
for years? How could this be different? 



Assessing the Research Paper 

Early in the spring of 1997 we decided to undertake an assessment of the senior seminar course. There were two 
reasons for making this suggestion. First, the senior seminar is the capstone in the general education curriculum of 
the College of Arts and Sciences. Second, the senior seminar guidelines mandated a research paper of substantial 
length. Thus, an assessment of outcomes focused on the research paper seemed both highly feasible and likely to 
yield significant information about how well instructors understood and were enforcing the requirement and how well 
students were doing in attempting to meet it. The Assessment Committee and the General Education Committee both 
thought this project worthwhile and agreed to authorize and cooperate with it. 
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The main aim of the project, asfirst conceived, was to see how consistently research papers were graded across different 
sections of the course. The method chosen was to take a sample of unmarked papers and have these regraded by 
members of the General Education Committee. Grades and regrades would then be compared. 



In the spring of 1997 thirteen sections of the senior seminar were offered. Each instructor was asked to submit an 
unmarked copy of every research paper from his or her section and to fill out and send in a form rating each of these 
papers and reporting the paper grade. The ratings form asked the instructor to check a box from one to five in the 
following categories: conceptual understanding, reasoning skills, research and bibliography, organization of the 
paper, and writing mechanics. A rubric defining the lowest (one) and highest (five) ratings in each category 
accompanied the form. Papers and ratings forms were received from eight sections (a total of 108 papers). 

In May 1997 the General Education Committee met and read a stratified random sample of these papers. The sample 
was constructed by sorting the papers into four piles by grade (A, B, C, and D-F); two piles were selected at random 
for each section and one paper was randomly selected from each of these two, for a total of sixteen. Selected papers 
were read, rated, and graded by a member of the committee. These ratings and grades were then compared with those 
given by the original instructor. 

Grades and regrades in this first study were remarkably consistent. Of the sixteen papers in sample, only two had 
grades and regrades that differed by two levels (e.g., A to C or B to D). (This pattern was not borne out in later studies.) 
Anotherfinding, which was both surprising and disturbing, was thateitherthe instructor or the regrader judged three 
of the papers in the sample (19 percent) very likely to have been plagiarized. 



This was to have been a full-scale assessment, but papers and forms were received from only four of the eight sections. 
The committee decided to go ahead with this as a second pilot study and to take steps to ensure greater compliance 
in the future. 

The ratings form and rubric were the same as in the previous study. However, in this case we sampled four papers 
from each section, one taken from each of the four grade categories: A, B, C, and D-F. Here for the first time we found 
some real differences in grading and rating of papers. Regraders assigned papers the same letter grade-ignoring 
pluses and minuses-in only five of the sixteen cases (29 percent). In two cases (12 percent) there was a difference 
of two grade levels. 

Since the first pilot study had not turned up serious discrepancies in grading, we thought that the discrepancies that 
turned up here might be an anomaly, due in part to the fact that the original instructors were basing the paper grade 
not only on overall student performance (reflected in the ratings grid) but also on process factors. A grade, for 
example, might be lower because the student did not comply with some specific direction of the instructor or because 
the student turned the paper in late; it could be higher because the instructor saw significant improvement or 
perceived some special virtue in the way the topic was treated. Regraders, on the other hand, would be unaware of 
the process and could base their judgment of the paper only on what they actually saw-the paper itself taken out 
of context. 

In order to control for these process issues, we decided to redesign the ratings form and rubric. On the new form, 
instructors were asked to rate the paper as before, but to give it two grades: one that reflected performance only and 
another that would take into account process issues, if any. This new ratings form was used in both of the subsequent 
assessments. 



First Full Assessment: Spring 1998 

The committee received papers and forms from five of six sections. As before, we took a stratified sample by grade 
category, four papers from each section. The results were not what was expected, and on the whole they were not 
very encouraging. For one thing, two of the twenty papers sampled (10 percent) were thought to have been 
olagiarized. One was so identified by the instructor, another by the regrader. 



First Pilot Study: Spring 1997 



Second Pilot Study: Fall 1997 
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Moreover, contrary to what we had expected, the consistency between the pure performance grade and the regrade 
showed no improvement over the previous study. The same letter grade— again, ignoring pluses and minuses— was 
assigned by both the instructor and the regrader in only five cases (25 percent of the sample). In other words, the 
regrader assigned the paper a different letter grade 75 percent of the time. Worse yet, there was a difference of two 
or more grade levels in nine cases or 45 percent of the sample. Two of these differences were due to the belief that 
the paper had been plagiarized, but this still left seven serious discrepancies (35 percent of the sample), and in all 
seven of these cases the higher grade had been assigned by the original instructor. 

A close look at these seven cases suggested that some instructors tended to be biased in favor of the students, 
particularly where the students in question strongly supported the “right” side of a controversial issue. But more 
important was the fact that instructors were accepting as research papers student compositions that were essentially 
expository or opinion pieces. Committee members, by contrast, tended to give such efforts low grades. 



Second Full Assessment: Fall 1998 

In the fall of 1998 papers and ratings forms were received from all eleven sections of the senior seminar. However, 
the sample selected for examination and regrading was constructed somewhat differently this time. The committee 
was now very concerned about discrepancies in grading. Since previous assessments had turned up major 
discrepancies almost exclusively in papers whose original grade was A, B, or C, they decided to exclude papers that 
had been given a D or an F by the original instructor. Because of the relatively large number of sections in this study, 
the com mittee decided to sample three papers from sections of twenty or more students, and two papers from sections 
with fewer than twenty. 

Overall results were similar to those of the previous assessment. Of a total of twenty-six papers sampled, discrepancies 
of two grade levels or more were found in eight cases, representing 31 percent of the sample. As before, where there 
was a major difference in grade, the regrades were the lower of the two. 

The committee looked closely at each of these cases. On the basis of this analysis, the committee reached two 
conclusions. The first is that instructors simply have different ideas about what constitutes a research paper. In most 
cases a committee member assigned a lower grade because the paper was seen as expository, a more or less 
competent summary of a body of facts, and thus judged to be a report and not a research paper. Committee members 
were unwilling to accept such papers as genuine research papers because the papers in question had no thesis, no 
argument, and no real conclusion. Some of the instructors, by contrast, were very willing to accept these same papers 
and in fact gave them good grades, particularly when the exposition was well-written and clear. 

In addition, plagiarism turned up once again. Although in this study the sample was constructed in a way that would 
exclude papers judged plagiarized by the original instructor (since instructors would give such papers a failing grade), 
nevertheless one paper in the sample was identified by the regrader as probably plagiarized. The original instructor 
had not suspected plagiarism; the paper had been given a grade of A. 



Conclusions of the Project 

We began this our first assessment study with the expectation that instructors would have more or less uniform 
standards and that grading across sections would be reasonably consistent. This belief had long been reinforced in 
discussions of grading with each other. Instead, we found an overwhelming lack of consistency in grading. Some of 
this could be traced back to normal and acceptable differences in weighting: for example, one instructor might 
consider reasoning skills as very important in determining the overall worth of a paper, while another might take 
writing mechanics or research and bibliography as paramount. But the vast majority of disparities in grading could 
not be so explained. Instead, they revealed some serious issues. 

0 Different standards. Most committee members believed that a research paper would have to be argumen- 
tative. However, we found that while some instructors demanded an argumentative paper, others were quite 
willing to accept competent expository papers as meeting the standard. 

Content grading. Some senior seminars were organized around political and moral issues, such as the 
politics of AIDS, the history of the Holocaust, and the treatment of Native Americans. Students whose papers 
showed a depth of understanding and sympathy for these issues were sometimes forgiven for less-than- 
adequate performance in writing. 
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0 Plagiarism. All of our studies turned up papers that had clear marks of wholesale plagiarism [all or nearly 
all of such papers had been copied or purchased]. In many cases the instructor had not seen the signs of 
plagiarism and had given the paper a high mark [A or B] without further investigation. 



Actions Taken 

During the last year, the General Education Committee has spent most of its time working on a complete revamping 
of our composition program. We have now finished and a reconsideration of the senior seminar is on the agenda this 
semester. Nevertheless, we have taken several actions to deal with issues of grading. 

0 Research paper requirement. After several sessions debating the definition of a research paper, a 
consensus finally emerged that a well-written expository paper could serve the needs of students in this 
course, and would be more appropriate for some topics. The requirement that the paper be argumentative 
was dropped; the guideline now says only that “The paper should involve original research.” 

0 Different weighting and content grading. This is a far more difficult issue. That instructors should have 
somewhat different values is normal; any coercion in this area would rightly be seen as a violation of academic 
freedom. However, there is no reason why we cannot help instructors to become more conscious of their own 
grading practices. We sponsored a workshop on grading last year, and we are considering requiring annual 
or semi-annual workshops for all general education faculty. Our hope is that these workshops will foster a 
community of instructors who agree on at least some of the elements that should go into the evaluation of 
student papers. 

0 Plagiarism. One of the clearest and most disturbing results of this assessment project was that there is a 
chronic low level problem of plagiarism, and that many instructors do not know howto detect plagiarism or 
what to do about it. We have had two faculty workshops on plagiarism to inform people about it. But we think 
that prevention is far better than detection and punishment. One of the best forms of prevention is to work 
closely with students while they are in the process of doing the paper. Students who are assured that they will 
receive plenty of help are less likely to plagiarize; moreover, asking students to turn in topics, bibliography, 
and parts of the written paper early in the semester simply makes it harder for them to engage in wholesale 
plagiarism. The senior seminar guidelines now require instructors to grade early drafts of the papers. The 
revised guidelines also require that there be at least one in-class writing assignment so that instructors will 
be aware of each student’s own writing abilities and vocabulary level. This, too, helps prevent plagiarism. 

It now seems obvious that we need to have much more ongoing training and supervision of instructors who teach 
the senior seminar and other general education courses. We are considering different ways of making this happen. 

Contrary to our initial assumption that assessment was exclusively about students and their work, we learned through 
these assessments of our practices as a faculty that there was a real need to improve our own performance as 
assessors of others. We believe that the process has resulted in benefits to students as well as to faculty and that our 
practice as professionals has been significantly enhanced. 
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General Education, Assessment, 
and Accreditation: 

Do Faculty Perceive Improvement? 

Mark E. Nutter 



Background 

General education has long been a topic of controversy at colleges and universities. Many, both inside and outside 
of higher education, have called for its improvement, but few have agreed that significant improvement has ever been 
achieved. As Cross wrote, “It is fairly easy to trace the fortunes and misfortunes of general education over the decades; 
it is far more difficult to evaluate its impact on education and society” 0982, p. 15). In the last ten years, accreditation 
agencies such as the North Central Association have adopted specific guidelines by which the centrality of general 
education programs may be evaluated. As part of this new emphasis, much of the accreditation process has shifted 
from a focus on the inputs of higher education to the outcomes of students’ learning in general education as well as 
the major area of study. 

Since the faculty deliver instruction, it is unlikely that even accreditation agencies can have much effect on changing 
the educational process without their support. However, much research into general education has focused on 
administrators or on students; few studies have looked at how faculty have been reacting to guidelines for general 
education that are tied to accreditation. 



Purpose of the Study 

This study was developed to provide information about community college faculty members’ attitudes toward the 
guidelines for general education established by the North Central Association’s Commission on Institutions of Higher 
Education. Specifically, the study attempted to determine whether a relationship existed between the Commission’s 
guidelines as outlined in the statement on “Documenting the Centrality of General Education” CNCA, 1 997, p. 24) and 
community college faculty members’ perception of satisfaction with the general education program at their colleges. 
The study also attempted to determine whether a relationship existed between those guidelines and community college 
faculty members’ perception of change in their general education program during the period from 1994 to 1999. 



Methodology 

The population studied was the approximately 21 ,300 full-time faculty members at the community colleges accredited 
by the North Central Association. The source of the population was a collection of college catalogs from 260 
[92 percent) of these institutions. The sample of 369 faculty was systematically selected from these catalogs, resulting 
in a sample that was stratified by college size., 

A questionnaire consisting of three sections: 10 demographic items, 34 Likert-type items, and 3 open-ended 
questions was developed by the researcher. The demographic items served to categorize the respondents in ways 
that might have a relationship to attitudes toward general education. The Likert items were organized in subscales 
that measured faculty attitudes toward [1) the individual aspects of the NCA guidelines for general education 
programs; [2) the combined aspects of the NCA guidelines for general education; [3) students’ preparation for 
college-level general education study; [4) perception of change in their college’s general education program over the 
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past five years; and [5] satisfaction with the general 
education program. The open-ended questions asked 
respondents to add additional information about the 
general education programs at their colleges. Two hun- 
dred and fifty-five faculty members [69 percent of the 
sample] responded to the survey. 

The SPSS statistical package [Base 9.0] was used to 
analyze the data through the application of non-para- 
metric statistics. The level of significance for all hypoth- 
eses tested was .05. Spearman’s Rho was used to test 
correlation hypotheses. The Mann-Whitney U test and 
the Kruskal-Wallis one-way ANOVA test were used to 
test the other hypotheses. A principal components analy- 
sis was also done to examine how the various subscales 
interacted. 



Research Questions 

Seven of the research questions focused on relationships 
between individual elements of the NCA’s guidelines for 
general education and [1] the faculty members’ current 
levels of satisfaction with their colleges’ general educa- 
tion programs, and [2] the faculty members’ perceptions 
of change in their colleges’ general education program 
over the last five years. Positive correlations for satisfac- 
tion with the general education program were found on 
the basis of [1] perception of general education as a part 
of the college mission, [2] perception of integration of 
general education throughout the curriculum of the 
college, [3] perception of faculty ownership of general 
education, [4] perception of faculty participation in gen- 
eral education, [5] perception of administrative support 
for general education, [6] perception of faculty involve- 
mentin evaluation of the general education program, and 
(7] perception of coherence of the general education 
program. Additionally, positive correlations were found 
for the same seven variables with respect to perception 
of positive change in the general education program over the preceding five years. All correlations were significant 
at the .05 level, and the null hypothesis was rejected in each case. 

Two research questions focused on the relationship between faculty satisfaction with their college’s general 
education programs and (l]the existenceof clearly defined goalsorobjectivesof general education, aswell as [ 2 ] the 
existence of an individual or committee to coordinate the general education program. Both tests were significant at 
the .05 level, and the null hypotheses were rejected. It appeared that the presence of each of these factors was related 
to faculty satisfaction with their general education programs. 

Research questions focusing on differences in faculty members’ level of satisfaction with their colleges’ general 
education program based on various levels of four demographic variables: age, length of employment, teaching area, 
and college size resulted in no relationships that were significant at the .05 level. The null hypothesis failed to be 
rejected in each case. 

Other research questions looked at [1] faculty members’ levels of satisfaction with their colleges’ general education 
program and [2] faculty members’ perception of change in their colleges’ general education program in the last five 
years in relationship to the time elapsed since their college’s last NCA comprehensive evaluation for accreditation. 
Neither test was significant at the .05 level, and the null hypotheses failed to be rejected. However, similar research 
questions focusing on satisfaction and change in relationship to the time elapsed since the last general education 
program review were significant at the .05 level. It appeared that colleges reporting having reviewed their general 
education programs within the last two years had the highest level of satisfaction and the highest level of perceived 



Documenting the Centrality of 
General Education 

It is essential that an institution of higher education 
seeking initial or continued affiliation with the 
Commission document and make public the 
centrality of general education to its educational 
endeavors. An evaluation team considers whether 
the institution’s 

o mission and purposes statements articulate 
the centrality of general education; 

o statements of educational philosophy demon- 
strate how general education goals are inte- 
grated into core, major, and elective courses 
within the major; 

o institution-wide general education learning ob- 
jectives are clearly articulated and publicized; 

o assessment of academic achievement includes 
the general education component of the pro- 
gram and is linked with expected learning 
outcomes; 

o faculty teaching general education courses 
hold graduate degrees that include substantial 
study (typically a minimum of 18 semester 
hours at the graduate level) appropriate to the 
academic field in which they are teaching; 

o faculty have ownership and control over the 
general education curriculum through active 
participation in appropriate governance struc- 
tures; and 

o facultysystematicallyand comprehensively re- 
view the general education curriculum. 

(NCA, 1997) 
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change. The two-year period was also the category with by far the highest number of responses, which may have also 
affected these results. 

Another research question examined faculty members’ levels of satisfaction with their colleges’ general education 
programs in relationship to approaches to assessment of general education. The test was significant at the .05 level, 
and the null hypothesis was rejected. It appeared that those respondents reporting that their colleges were using a 
locally developed general education test, a combination of assessment methods, or course-based methods were 
generally more satisfied with their general education programs than were those who reported using a commercially 
developed test or no formal procedure for assessment. A similar comparison found no significant connection between 
satisfaction with the general education program and the method used to deliver general education. 



Additional Findings 

□ NCA Guidelines Seem to Be Effective 

While it is true that correlation should not be interpreted as causation, the strength and the number of the positive 
correlations found between NCA guidelines for general education programs and the measures of satisfaction and 
perceived positive change in general education programs appear to be meaningful. One possible conclusion that 
can be reached from these findings is that the NCA guidelines are working, at least in terms of faculty satisfaction 
with and perceived improvement in general education programs. Three points are of particular importance. 

o Each of the elements of the NCA guidelines for general education was positively and significantly 
correlated with both satisfaction with the general education program and perceived positive improve- 
ment in the general education program. 

o This finding was further supported by a strong positive correlation of .621 (significant at the .01 level) 
between the combi ned elements of the NCA guidelines and faculty’s perceptions of satisfaction with their 
general education programs. 

o The combination of NCA guidelines had a stronger correlation with perceived improvement i n the college's 
general education program (.598, significant at the .01 level) than did any single element of the guidelines. 

These findings support NCA’s assumptions about the indicators of a strong general education program, and also indicate 
that improvement is more strongly associated with the combination of those factors than with any single factor. 

□ General Education Goals Seem to Be a Significant Factor 

The community colleges accredited by the NCA have made significant progress in establishing goals or objectives 
for general education. Nearly 92 percent of faculty responding to the survey indicated that their colleges had 
established such goals. 

The importance of general education goals was also reinforced through responses to the open-ended survey 
questions. The existence of clear goals or objectives was cited by 20 respondents as a strength of their colleges’ 
general education program. Likewise, lack of clear mission, goals, or expected outcomes was frequently cited as 
a weakness. 

□ Leadership or Coordination for General Education Is Related to Satisfaction and Coherence 



0 

ERIC 



The community colleges accredited by the NCA have made considerable progress in establishing leadership for 
their general education programs. More than 77 percent of the faculty members responding to the questionnaire 
indicated that their institutions had an individual or committee to coordinate the general education program. This 
would seem to indicate that a change has taken place since Gaffs (1993) observation that general education 
generally lacks leadership. The existence of an individual or committee to coordinate general education was found 
to be positively related to both satisfaction with the general education program and coherence in the general 
education program. These findings support the observations of Hinson and Stillion (1996), who indicated that the 
appointment of a committee and/or director for general education was related to improvement in general 
education programs. It is also interesting to note that many of the respondents' comments about general education 
program weaknesses were specifically problems requiring coordination and leadership to resolve: lack of 
consistency, failure to use data to improve general education, and lack of integration of general education 
throughout the curriculum. 

101 



96/ A Collection of Papers on Self-Study and Institutional Improvement 2001 



□ Diversity in Approach 

The NCA Commission on Institutions of Higher Education (1997, p. 24) has stated that approach to general 
education is of secondary importance to the creation of a coherent program. It is evident that considerable diversity 
in approaches to general education does exist at community colleges accredited by the NCA. In addition to five 
common approaches listed on the survey, 58 respondents identified a total of 1 8 combinations of those approaches 
being used at one or more colleges. In fact, a combination of approaches was the most common method of delivery 
and was reported by 23.1 percent of the respondents. The combination approach was followed closely by 
distribution requirements with a wide choice of courses (21.2 percent). 

At the same time, there was no evidence that any particular approach was more related to satisfaction than any 
other. This would seem to indicate that faculty would do well to find an approach that fits the needs of their 
institution and not try to find a “best” method. 

□ Diversity in Assessment 

While general education has existed since the early part of this century, the concept of formal assessment of 
general education is relatively new and still under development on many campuses. In fact, nearly half (44.3 
percent) of the respondents reported that their colleges were relying solely on course-based assessment to judge 
the effectiveness of general education programs. Beyond that, there was considerable diversity in approaches to 
assessment of general education. In addition to five common assessment methods listed on the survey, 54 
respondents identified a total of 13 combinations of those approaches being used at one or more colleges. 
However, it was evident from respondents’ comments that assessment was not institutionalized at many colleges. 
Respondents comments included: “We’re still working on assessment procedures.” “Assessment could be 
improved.” “No one is sure what the term general education means nor does anyone know howto assess it.” “Some 
outcomes are very hard to assess.” “Each faculty is assessing individually at this time.” “We haven’t done much 
other than talk.” 



It was also evident that faculty were less pleased with commercially produced tests than they were with a 
combination of assessment methods, a locally developed test course-based assessment, or “other assessment 
methods.” In fact, the use of a commercially produced test even ranked below having no procedure at all for 
assessing general education skills. Additionally, those faculty who were likely to give high marks to their institutions 
in the area of completing the feedback loop for general education assessment were also likely to report that their 
college used a combination of assessment techniques. 

□ Coherence Is Closely Related to Satisfaction 

The attribute most highly correlated with satisfaction with the general education was coherence (.81). Those who 
reported thattheir colleges’ general education programs were coherent, organized, and logical were also very likely 
to be satisfied with the programs. The strength of this correlation indicates that coherence is a particularly 
important factor in relationship to faculty members’ satisfaction with their college’s general education programs. 

□ Constant Review 



Review and reform of general education seems to have been a focus on campuses for at least two decades now. 
The study found more than two-thirds of the respondents indicating that their college or program had reviewed 
the general education program within the last two years. 

□ Comments Indicate Change 




Significant change does seem to be occurring. The fact that 1 70 respondents chose to add written comments about 
the strength of their colleges’ general education programs and that 153 took time to comment on program 
weaknesses can be viewed as an indicator of the current attention paid to general education reform. It is also worth 
noting that the comments, which came from a cross-section of full-time faculty, reflected many of the aspects of 
the NCA guidelines for general education. The comments seemed to indicate that a good deal of discussion and 
work was occurring in the area of general education as it relates to accreditation. For example, one respondent 
wrote, “As a result of (the) NCA visit we have carefully spelled out our general education outcomes. There is an 
effort to teach and assess those outcomes.” Another respondent wrote, “We have two committees that assess the 
success of meeting general education goals on an on-going basis. I’m on one of the committees. Strong program.” 
Another respondent emphasized the ongoing nature of general education development: “We are constantly re- 
evaluating what we are doing and why. We work with the individual.” 
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□ Dissatisfaction with Student Preparation Is Not Related to Faculty Satisfaction with General Education 

While faculty members seemed somewhat frustrated by the level of preparation of their incoming students, that 
frustration did not seem to be significantly related to their level of satisfaction with their colleges’ general education 
program, at least in the quantitative portion of the study. The frustration did surface, however, in the qualitative 
portion through the respondents’ comments, and it was the second most frequently cited program weakness. The 
weakness seemed to lie more in pre-enrollment testing and placement than in the actual general education 
component of the curriculum. Comments such as these were common: “Because of our open door policy, many 
students do not enroll in remedial classes prior to taking college level classes.” “The only problem is that they’re 
taking developmental studies at the same time they’re taking their major courses.” “We lack mandatory placement 
in remedial courses, so our students often are not prepared with needed background knowledge.” “Some student 
who have educational handicaps are allowed into classes where they may fail because they lack some basic skills 
necessary for success.” 

□ Time Elapsed Since General Education Review 

Recency of general education program review seemed to be related to both satisfaction with the general education 
program and perceived positive change in the program. This finding may have been related to the disproportion- 
ately large number of individuals who reported that their colleges had reviewed their general education program 
within the last two years. Perhaps the immediacy of that review led to a more positive view of the program, or 
perhaps the requirements of the NCA are causing program reviews to become more focused and outcomes-based. 



Summary and Conclusions 

This study demonstrated that there is a strong positive correlation between (1) the presence of the elements of the 
NCA’s guidelines for general education programs and faculty satisfaction with their colleges’ general education 
programs, and (2) the presence of those same elements and faculty members’ perception of positive improvement 
in their colleges’ general education programs over the past five years. Although each of the elements of the NCA’s 
guidelines was positively correlated with satisfaction and change, the combination of those elements had among the 
highest correlations with both satisfaction with the general education program and perceived positive change in the 
general education program over a five-year period. 

Satisfaction with the general education program appears to be positively related to the existence of college-wide 
general education goals and the presence of an individual or committee to coordinate the general education program. 

Satisfaction with general education appears to be unrelated to level of preparation of incoming students, teaching 
area, college size, time elapsed since the last NCA comprehensive evaluation, and the college’s approach to delivery 
of general education. 

There appears to be a relationship between faculty satisfaction with the general education program, time elapsed 
since the last general education program review, and the method of assessment used for general education. 
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Measuring Spiritual Change: 
Initial Findings 



Will Slater 

The Faithful Change Questionnaire (FCQ) is made up of eight of the more promising paper and pencil measures of 
spirituality (see Table 1 for list of the measures included in the FCQ). The FCQ data set was not yet complete as of 
this writing. However, 31 6 FCQ questionnaires from five of the eight participating institutions have been received, and 
the following preliminary analyses were conducted. The intercorrelation matrix of the total scale and subscales of the 
instruments that make up the FCQ is presented in Table 2. There are a number of interesting findings. Several clusters 
of subscales display relatively high intercorrelations. The various measures of positive religious belief or experience 
form one cluster. Measures of spiritual maturity developed from liberal/universalist (Genia), mainline Protestant 
(Benson], and conservative Evangelical (Ellison; Hall & Edwards] theological perspectives correlate positively and 
significantly with each other. Fundamentalism also correlates significantly and positively with these measures. 

Quest and other measures of openness to the complexities of existential questions form a second cluster consisting 
of Batson’s (BQust] and Altemeyer’s Quest (AQust], Dudley’s Religious Maturity Scale (RMS), and Genia’s Openness 
Scale (SEI-SO). BQust and AQust scales, which ostensibly measure the same construct, share only 22 percent 
common variance. The scales measuring openness and doubt correlate negatively with the positive religious 
experience measures in the first cluster. 

One construct that is difficult to place in either of these two clusters is Benson’s measure of horizontal religious 
commitment (FMS-H).This subscale, which involves a commitment to the social justice aspects of the gospel, displays 
modest positive correlations with several measures in both the positive religious experience cluster and the openness 
and doubt cluster. 

All of the 17 correlations between social desirability and the various religious measures are below 0.20, and only three 
are statistically significant. 

An analysis of variance (see Table 3) was conducted to allow cross-sectional comparisons of first-year students and 
seniors. Several measures discriminate significantly between first-year and senior students: AQust (p< .000), the 
Spiritual Assessment Inventory’s Realistic Acceptance (SAI-RA) (p< .020), Benson et al.’s Horizontal (FMS-H) (p< 
.01 6) and Benson et al.’s Vertical (FMS-V) (p< .053), Altemeyer and Hunsberger’s Religious Fundamentalism (RFS) 
(p< .001), Dudley and Cruise’s Religious Maturity (RMS) (p< .001), Genia’s Openness Scale (SEI-SO) (p< .001), and 
her Spiritual Experience Index (SEI) (p< .009). Examination of the means in Table 3 reveals that AQust, RMS, FMS- 
H, SEI-SO, and SEI all were higher for seniors. However, SAI-RA, FMS-V, and RFS were all lower for seniors. 



Discussion 

The intercorrelations of the various measures are consistent with their content as revealed by the coherent clusters 
of subscales. It is encouraging to find that measures of spiritual experience/belief from diverse theological 
perspectives correlate with each other significantly and positively. Openness instruments like AQust and BQust, and 
related constructs like SEI-SO and RMS, with its open-minded commitment aspect, correlate moderately and 
positively with each other and negatively with the measures of positive religious experience. These relationships may 
reflect the tension inherent between openness and doubt versus depth of positive religious experience. The 
dimensionality underlying the openness instruments-particularly AQust and BQust, which were intended to measure 
the same construct (but account for only 20 percent common variance)-needs to be investigated and perhaps 
subscales need to be developed. It is interesting that neither Quest scale (discounting the small 0.14 correlation of 
AQust) is related to the FMS-H dimension, a measure of one’s commitment to the social justice aspects of the gospel, 
with its emphases on rejecting racism and concern for the poor. However, both the spiritual experience and belief- 
oriented scales and the other openness measures do correlate with FMS-H. The openness instruments have their 
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highest negative correlations with fundamentalism. Fundamentalism traditionally correlates positively with prejudice 
and is considered to be a better predictor of prejudice than the original Intrinsic/Extrinsic religious orientation 
constructs. 

It was a pleasant surprise to find very little influence of social desirability on these measures. Still, the influence of 
social desirability should be investigated with the full data set to determine if one or the other of the underlying 
dimensions of social desirability may influence scores on some of the instruments. It may be necessary to control for 
social desirability on those instruments where the influence, even if modest, is still significant. 

In this cross-sectional sample regarding faith development, seniors scored significantly higher than first-year 
students did on the openness dimension and on FMS-H [horizontal religion), but lower on two of the positive religious 
experience scales and on RFS (fundamentalism). These data, when viewed in conjunction with the correlations among 
measures discussed earlier, suggest a complex picture regarding faith changes over the college years. It may be that 
the existential doubt reflected by the Quest concept is something that results from four years of liberal arts education. 
A broadening of one’s perspective may be the positive benefit of such an education at the expense, perhaps, of one’s 
certainty with regard to personal faith matters. However, such broadening may not always lead to a more socially 
responsible orientation (as reflected in the lack of correlations with the FMS-H). It may be that Quest, with its inherent 
existential doubt, may undercut one’s moral basis for caring (assuming FMS-H taps caring in some way). On the other 
hand, religious beliefs do seem to be related to FMS-H; those who retain higher levels of religious belief, either the 
more Evangelical (SAI and SMI) or more liberal (SEI-SS),a!so retain a moral basis for their social concern, some degree 
of certainty about God. 

Is it the case that RMS (a open form of religious maturity) and SEI-SO (openness) reflect a religion that retains the 
moral basis for caring (some degree of certainty about God) even in the presence of theological uncertainty (spiritual 
openness)? Do the scales like SAI-A, SMI, and SEI-SS tap social consciousness (traditionally a liberal ideology) in the 
context of a personal faith that is undergirded by more theological certainty? In other words, these scales may capture 
two somewhat different religious orientations: (1) more liberally oriented, socially sensitive persons whose faith is alive 
in the context of doubt; and (2) more conservative, Evangelical persons who are also socially sensitive but more theologically 
certain. Quest may tap the doubt component at the expense of the moral belief component, and RFS may tap the religious 
certainty component at the expense of social sensitivity (both have negligible correlations with FMS-H). 

This conceptualization is admittedly a speculative extrapolation from the correlational data; but if it is supported by 
the longitudinal and qualitative data, this would clearly have some meaningful implications for liberal arts education 
in the context of religious institutions. Liberal arts education may foster the opening up of one’s belief system, creating 
a broader perspective. The religious institutional context seeks to foster the development of a coherent religious 
worldview. In the ideal outcome, the student emerges with a broader perspective on the God in whom he or she 
believes and has a moral sense that informs social sensitivity. Openness and doubt without a moral framework 
becomes skepticism and disenfranchisement; religious belief without openness becomes insensitive fundamental- 
ism. The data to date suggest tension but also synergy between the goals of liberal arts education and religious 
contextualization of that education. 

Clearly, the inverse relationship between the openness and doubt cluster and the positive spiritual experiences cluster 
needs further study in an effort to understand the role each of these aspects plays in spiritual/faith development. This 
is a very important issue that demands further study. Since the current data are cross-sectional, confirmation of these 
conceptualizations awaits the longitudinal data we are proposing to gather. We will strive to determine whether the 
apparent decline in the spiritual belief/experience dimension is a true and lasting one or whether it is a forerunner 
of a greater and more authentic maturity developed by the students as they wrestle with the difficult existential issues 
of life. Perhaps out of this struggle they develop a spirituality/faith that is their own, that cares about the plight of the 
poor and others in need, and that can better weather life’s storms. 



Quantitative Work Team: Keith Edwards, Rosemead School of Psychology 

Todd Hall, Rosemead School of Psychology 
Pam Nath, Bluffton College 
Allan Oda, Azusa Pacific University 
Brian Eck, Azusa Pacific University 
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Measuring Spiritual Change: 

Initial Findings 

Table 1. 

Measures in the Faithful Change Questionnaire 



Abbreviation 


Instrument 


Author 


AQust 


Quest 


Altemeyer & Hunsberger 


BQust 


Quest 


Batson & Schoenrade 


SAI 


Spiritual Assessment Inventory 


Hall & Edwards 


SAI-A 


Awareness [sub-scale) 


Hall & Edwards 


SAI-D 


Disappointment [sub-scale) 


Hall & Edwards 


SAI-G 


Grandiosity [sub-scale) 


Hall & Edwards 


SAI-I 


Instability [sub-scale) 


Hall & Edwards 


SAI-RA 


Realistic Acceptance [sub-scale) 


Hall & Edwards 


FMS 


Faith Maturity Scale 


Benson et al. 


FMS-H 


Horizontal [sub-scale) 


Benson et al. 


FMS-V 


Vertical [sub-scale) 


Benson et al. 


RFS 


Religious Fundamentalism Scale 


Altemeyer & Hunsberger 


RMS 


Religious Maturity Scale 


Dudley & Cruise 


SDB 


Social Desirability 


Marlowe-Crowne 


SMI 


Spiritual Maturity Index 


Ellison 


SEI 


Spiritual Experience Inventory 


Genia 


SEI-R 


Spiritual Experience Inventory - Revised 


Genia 


SEI-SO 


Spiritual Openness [sub-scale) 


Genia 


SEI-SS 


Spiritual Support [sub-scale) 


Genia 
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Measuring Spiritual Change: 

Initial Findings 

Table 3. 

Cross-Sectional Comparisons Means, Standard Deviations, 

and Anova Results 



Instrument 


Classification 


N 


Mean 


SD 


F 


Sig. 


AQust 


First Year 


145 


49.26 


7.68 


13.12 


0.000 




Seniors 


73 


53.71 


10.09 






BQust 


First Year 


144 


41.47 


4.85 


1.43 


0.233 




Seniors 


73 


42.30 


4.89 






SAI-A 


First Year 


145 


3.84 


0.70 


3.39 


0.067 ] 




Seniors 


73 


3.64 


0.87 






SAI-D 


First Year 


145 


2.61 


1.03 


0.60 


0.441 




Seniors 


73 


2.50 


1.00 






SAI-G 


First Year 


145 


1.67 


0.62 


0.95 


0.332 




Seniors 


73 


1.59 


0.48 






SAI-I 


First Year 


145 


2.20 


0.81 


0.17 


0.676 




Seniors 


73 


2.15 


0.85 






SAI-RA 


First Year 


144 


4.33 


0.77 


5.47 


0.020 




Seniors 


73 


4.05 


0.88 






FMS-H 


First Year 


145 


3.93 


0.59 


5.95 


0.016 




Seniors 


73 


4.15 


0.67 






FMS-V 


First Year 


145 


5.09 


0.63 


3.77 


0.053 




Seniors 


73 


4.90 


0.77 






FMS 


First Year 


145 


4.43 


0.43 


0.23 


0.630 




Seniors 


73 


4.40 


0.52 






SDB 


First Year 


117 


12.31 


3.88 


1.53 


0.217 




Seniors 


65 


13.03 


3.57 






RFS 


First Year 


145 


79.99 


10.89 


11.08 


0.001 




Seniors 


73 


74.10 


14.80 






RMS 


First Year 


145 


39.41 


7.61 


12.34 


0.001 




Seniors 


73 


43.21 


7.34 






SEFSO 


First Year 


144 


32.26 


7.05 


0.93 


0.337 




Seniors 


73 


35.56 


6.45 






SEFSS 


First Year 


144 


66.22 


8.71 


1.92 


0.167 




Seniors 


73 


64.90 


10.86 






SEI-R 


First Year 


144 


98.48 


9.46 


6.96 


0.009 




Seniors 


73 


100.47 


10.99 






SEI 


First Year 


1 44 


158.70 


13.16 


1.38 


0.242 




Seniors 


73 


163.75 


13.65 






SMI 


First Year 


144 


134.27 


17.69 


11.22 


0.001 




Seniors 


73 


131.00 


22.41 
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Learning Community @ 
Capella University 



Alien Jaisle 
Scott Lindgren 
Michael Petkovich 



This presentation will share the experience of Capella University in establishing the Learning Community @ Capella 
University, a web-enabled system to encourage and promote the development of a learning community to enhance 
online learning. The learning community includes several elements: learner web sites to showcase their learning 
projects; learner and faculty discussion forums; academic advising communications; team-based cooperative 
learning; and support centers, such as the development of a Thinking Habits Center, which will focus attention and 
dialogue on learning to develop first-rate thinking habits equal to the challenges of the twenty-first century. 



The Challenge of Creating a Learning Community 



A key challenge common to online learning is the social isolation of learners from each other, from faculty, and from 
other supporting resources. Put yourself in the place of an online learner who is miles and worlds apart from other 
learners, faculty, and support services. 

o You feel like you are on your own, just you and your computer, struggling to learn a new subject in an unfamiliar 
learning format. 

o You feel so alone facing a major learning challenge, and you cannot easily have a two-way conversation with 
your classmates or your teacher. 

o When you get stuck, you wonder whom you can ask for help. You do not know what confidence you can place 
on the advice of people you do not know well. 

o You wonder whether others can understand your problem and whether they can actually help you. You may 
think you are the only one who does not understand. 

o Finally, when at long last you find a solution, you may feel that there is no one to appreciate and celebrate your 
success. 



Capella University is developing the Learning Community @ Capella University premised on the conviction that: 
learning is a social experience requiring a mutually supportive community of learning. The presentation will address 
Capella University’s thinking and experience with developing the theory and practice of an online learning community 
in our new School of Undergraduate Studies. Building community in online learning is a common challenge for every 
program of online learning, so the presentation will also provide a forum for engaging all participants in identifying 
learning community issues and promising solutions. 



Issues to Be Addressed in the Session 

o Why is learning a social experience requiring a mutually supportive community of learning? 
o What are the unique social isolation problems inherent in online learning? 
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o What are the unique community bui/ding opportunities inherent in online learning? 

o Whathas Cape/fa University's experience been in building the web-enabled learning communityfortheSchool 
of Undergraduate Studies? 

o How can cooperative learning contribute to and benefit from online learning community building? 

o What has been the experience of session participants with both the problems and opportunities of community 

building in online learning? 

Learning as Social Experience Requiring a "Learning Community" 

There has been a growing recognition that learning is a social experience requiring a supportive learning community, 
especially for online learning. Building Learning Communities in Cyberspace by Palloff and Pratt and Facilitating Online 
Learning by Collison, et al. f are excellent books that address learning communities online. 

During the development of the School of Undergraduate Studies, we at Capella University asked ourselves the two 
fundamental questions that relate to learning communities: 

o What is learning? 
o What is community? 

Our tentative answers to these questions are: 

The purpose of a learning experience in a course or program is to enable learners with the competencies to join the 
conversation of and to actively participate in a living community of practice [Curriculum as Conversation by Apple bee, 
and Communities of Practice by Wenger). 

Community is a dynamic whole that emerges when a group of people share common practices, are interdependent, 
make decisions jointly, identify themselves with something larger than the sum of their individual relationships, and 
make a long-term commitment to well-being (their own, one another's, and the group's ) ( Creating Community 
Anywhere by Shaffer and Anundsen). 

The image we have of learning community is that of a series of concentric circles representing larger and larger 
communities of conversation and communities of practice. The innermost circle begins with the learner, expanding 
to the virtual team community, the virtual classroom community, the university community, and the community of 
professional practice. 

The ultimate learning objective is to prepare the learner to participate in the outer circle of the community of 
professional practice. To do this we must enable the learner with the foundational domain knowledge and 
professional habits needed to enter the professional community of practice. Once in this outer circle of the community 
of professional practice, the learner can continue learning through learning conversations and learning practice as 
the profession evolves-that is, providing we have prepared the learner with a solid knowledge foundation and good 
professional habits. Of course, learners will periodically need to engage in a formal learning experience to enable them 
to shift from one community of practice to another. 

Online Learning Isolation and Community Building Opportunities 

Online learning has both isolation problems and community opportunities inherent in the dispersed networked 
technology it is based on. While learners are not physically near each other, they are connected through the relational 
technology of the Internet. Our challenge is how to make the most of the relationship building potential of this 
technology. 

The community building experience explored by the School of Undergraduate Studies at Capella University includes 
the following: 

0 Learner web site and e-portfolio of learning. Each learner develops a personal learning web site within 
the Learning Community @ Capella University, a set of learner, faculty, staff, and learning center web sites 
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hosted by Capella University. This gives learners the opportunity to showcase their work as a way of 
communicating with the community of professional practice they wish to enter into or to advance in. 

0 Professional learning projects. A learner’s work consists primarily in producing projects that reflect the 
professional practice of the professional community the learner wishes to join. This is active learning by doing 
the work of the profession, rather than just learning about the work of the profession. 

0 Team-based learning. Learning projects are both personal and team-based. On team-based projects, 
learners learn to collaborate in a small five-person work team reflecting professional practice. This requires 
overcoming not only the hurdles of effective collaborative teamwork s general, but it also requires mastering 
the challenges of virtual teamwork. 

0 Active cooperative learning. The learning strategy of the School of Undergraduate Studies at Capella 
University is premised on active cooperative learning [as pioneered in Active Learning by Johnson, Johnson, 
and Smith). The challenge is to translate the classroom-based active cooperative learning to the virtual online 
learning process. We will be presenting our tentative findings on what these challenges are and how to 
overcome them. 

0 Professional habits. An additional challenge that we have set for ourselves at Capella University is to develop 
in learners powerful professional habits, at the same time they learn the foundational domain knowledge for 
the community of practice they are preparing for. We have developed ten Thinking Habits of Mind, Heart, and 
Imaginationio provide professional habits to learners. These are being translated into rubrics for coaching and 
grading learners, along with the coaching and grading of the mastery of the domain knowledge. We have 
found that mastering the language of the Thinking Habits is the first step to mastery of the professional habits 
themselves. 

0 Academic advising, virtual lab assistance, and discussion forums. We have discovered that multiple 
ways of connecting to learners are important in reducing isolation and enriching the sense of supportive 
community. First, an academic advisor is in communication in multiple ways in the early stages of the learner’s 
degree program. This includes facilitation of a group discussion forum for new learners. Other discussion 
forums outside of the normal class dialogue are being explored aswell, including discussion forums for faculty. 
The Information Technology Program is developing a virtual laboratory environment as an analog to the on- 
campus computer laboratory. We are also exploring ways of supporting learners with virtual lab assistance. 



Format and Style of the Presentation 

Allen Jaisle, Associate Dean, will present an overview of the learning goals and architecture of the School of 
Undergraduate Studies, which requires the development of an effective and supportive community of learners. Scott 
Lindgren, Manager of eLearning Systems, will describe and demonstrate the web-enabled Learning Community @ 
Capella University. Mike Petkovich, Lead Instructional Designer, will engage session participants in a cooperative 
learning experience to identify and share their perspectives on the problems and opportunities of online learning 
community building. 



Active Learning: Engaging the Participants 

To put the online learning community experience of Capella University in a larger context in common, the session 
participants will be engaged in an active learning experience that involves a structured cooperative learning 
experience to (1) identify the perspectives of participants on the social isolation problems and opportunities inherent 
in online learning, and (2) share the experience of participants with building learning communities for online learning. 



Allen Jaisle is Associate Dean of the School of Undergraduate Studies at Capella University in Minneapolis. 
Scott Lindgren is Manager of eLearning Systems at Capella University in Minneapolis. 

Michael Petkovich is Lead Instructional Designer at Capella University in Minneapolis. 
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Institutional Support for 
Distance Learning: 
Lessons Learned in the Context of 
Recent Accreditation Guidelines 



Gregory Sojka 
Jian R. Sun 

Distance learning (DL) has become an important part of higher education’s strategic planning to provide courses to 
working, site-bound students. This growing interest and involvement in DL by more and more institutions of higher 
education also make educators rethink and reevaluate the pedagogy and practice in their teaching. The increasing 
scrutiny of quality control on distance learning from both the public and the accreditation agencies will require even 
higher standards for DL. Institutional-initiated support for involving faculty in designing and developing online 
courses becomes crucial, as has been recognized by many institutions, in making such strategic planning successful. 
This belief has now been reinforced more emphatically in the forthcoming guidelines currently still circulated in their 
draft format from the eight accreditation bodies on the evaluation of distance learning, which emphasizes the 
institutional support for a successful distance learning program. This document emphasizes adequate and 
appropriate staffing and technical assistance, an ongoing program of appropriate technical design, and production 
support for participating faculty members CNCA 2000). As one of the many institutions in the nation that have been 
engaged in the utilization of Internet technology to enhance distance learning, the University of Rio Grande has made 
tremendous efforts in initiating and sustaining faculty development workshops on the development of Internet 
courses. Lessons learned from our efforts will be of particular value to other institutions in their efforts to develop their 
DL courses and programs in terms of planning, organization, content, evaluation, technical support, faculty selection, 
objective feedback, and modification for future faculty training workshops. 



Background 

The administration started the initiative to conduct faculty workshops on distance learning course development in 
1999. All faculty were invited to submit proposals to design and develop online courses, which were reviewed by a 
special committee to select participants for the first distance learning workshop. Seven faculty from nursing, English, 
music, education, and technology were selected and attended a week-long workshop in the summer of 1999 
conducted by campus computing staff. Topics of basic computer technology, Internet skills, web page design, and 
the courseware platform, Blackboard, were covered. Participating faculty received a computer, a printer, and a 
scanner, provided by the institution. Upon completion of their courseCs), they would also receive a development 
stipend. A four-credit course would pay a $1000 stipend. Some 20 courses were proposed and designed by those 
participating faculty. Several were used either as web-based or web-enhanced courses in the following academic 
year, 1999-2000. The following summer of 2000 saw the second such faculty workshop. Six additional faculty 
representing sociology, English, philosophy, medical technology, and nursing participated in the second workshop. 
This workshop included a new courseware platform, WebCT, replacing Blackboard. Currently, some 20 additional 
courses are being developed using this new courseware platform. Lessons learned in the process of engaging faculty 
in the distance learning endeavor in the past two years include the following four aspects: 

o The need for strong institutional support 
o The need for appropriate evaluation of faculty workshops 

o The need for more planning and considerations of other issues involved in e-course development 
o The need to utilize our own faculty trainers for faculty workshops. 
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The Need for Strong Institutional Support 

The administration must provide leadership as well as technology support. Organizing faculty workshops in DL, 
inviting faculty and screening applicants, developing agenda, and securing funding for necessary equipment 
purchase cannot be overemphasized. When the administration makes such efforts and provides adequate support, 
faculty feel more motivated and willing to participate. The two institutional Summer Institutes had thirteen faculty 
member participants, 16 percent of the faculty. We hope that the third Summer Institute will attract even more faculty. 

The Need for Appropriate Evaluation of Faculty Workshops 

The success of a faculty workshop needs to be evaluated against a set of pre-articulated workshop outcomes. Our 
first faculty DL workshop did not receive adequate evaluation. Faculty attended the workshop, listened to the technical 
staff members explain selected technical features, and began to work on their own courses. No feedback of 
effectiveness or usefulness of the workshop presentation was collected. Such inefficiency and lack of evaluation were 
later recognized, and changes were made for the second workshop in both the content and format. The second 
workshop, however, was evaluated in both content and format. Without evaluation, faculty would flounder and DL 
workshops would fail. 

The Need for More Planning and Other Issues Involved in e-Course Development 

Participating faculty raised other issues involved in e-course development, such as marketing online courses, 
enrollment minimums and maximums, and an online course designation in the university database, all of which called 
for more consideration by the school administration. 

The Need to Utilize Our Own Faculty Trainers for Faculty Workshops 

Normally, faculty workshops on DL are conducted by campus computer staff whose technology expertise and 
knowledge seem to make such people qualified trainers. This belief creates a misperception that e-courses simply 
transform classroom materials into online teaching. Actually, faculty who have more experience in using such 
courseware programs can offer more sound e-pedagogical help to their colleagues than can the technical experts. 

We must develop new theories regarding student learning in this technology-enhanced environment. This new 
learning environment, appropriately called e-learning, displays unique elements that characterize the teaching and 
learning processes: e-interactivity, e-assessment, e-course design, e-delivery, and e-class management. 

Future workshops should utilize experienced faculty as faculty trainers well versed in e-pedagogy. Currently, a series 
of faculty-initiated mini-workshops has been planned and offered at our campus, facilitated by a faculty member. 
Each weekly workshop focuses on technical skills and pedagogical issues. Evaluation of the effectiveness of each 
workshop will be considered in planning the next workshop. 

This session would be of particular interest to those who are directly involved in web-based/enhanced courses: 
administrators, assessment directors/coordinators, faculty, and campus technology support staff. Other institutions 
involved in DL could avoid our mistakes and make better progress on their campuses. Since e-learning is still in its 
infancy, audience participation could contribute to our session. Such sharing among session participants can be 
developed into future topics of interest at next year’s NCA Annual Meeting. 

References 

NCA. 2000. Draft guidelines for the evaluation of electronically offered degree and certificate programs 
<http://www.ncacihe.org/resources/draftdistanceguide/introddg.html>. 
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Benchmarking Best Practices in 
Online Learning Communities 



Helen R. Connors 
Diane M. Billings 
Diane J. Skiba 
Robin Etter Zuniga 



During the past few years, institutions of higher education have committed billions of dollars to the use of advanced 
telecommunications technology for distance or distributed learning. Today, the most popular of these technologies 
is web-based education. As this on-line learning marketplace expands, both supporters and critics are calling for 
outcome data related to the success of these new technologies and their impact on teaching and learning. One 
process for establishing and improving quality outcomes is benchmarking. 



Best Practices and Benchmarking 

Benchmarking has gained popularity in the business sector as a research approach that discovers “best practices" 
in whatever process is designated for study. It is a process improvement technique that provides factual data to allow 
institutions to compare performance on specific variables in order to achieve best performance. Although this process 
is employed routinely in the business world, health care, and nursing service, it is infrequently adopted in the higher 
education arena. 

Since spring 1999, three large state institutions that span the central part of the United Stated have collaborated on 
a benchmarking project to determine best practices in on-line learning communities for nursing courses. Represen- 
tatives from the three institutions, partnered with a consultant from the Flashlight Program [www.tltgroup.org/ 
programs/flashlighthtml) to develop, implement, and evaluate an instrument to assess outcomes of web-based 
nursing courses. 



Benchmarking Process 

The first step in the benchmarking process is to determine what to benchmark. The variables selected for this 
benchmarking project were derived from frameworks and models used in nursing and higher education to illuminate 
the impact of the use of technology. 

The next steps are to enlist partners and define a mapping process. The three schools involved in this study 
collaborated with the Flashlight Program team to ensure that the research would provide a broad base of potential 
practice differences that could be revealed in the benchmarking process. The mapping process was based on the 
use of the technology, the course development process, and design for curriculum outcomes that are common to all 
three schools of nursing. 



Developing the Survey Instrument 



The instrument used to collect the benchmarking data for this study was adapted from the Flashlight Program Current 
Student Inventory [CSQ Toolkit. The CSI is designed to assess the student's views of technology-based teaching and 
learning [Ehrmann and Zuniga, 1997). It consists of a series of questions clustered into fourteen themes or subscales 
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that focus on the various reasons for using technology to enhance learning outcomes. These fourteen principles 
incorporate and expand the “Seven Principles of Good Practice in Undergraduate Education” developed earlier by 
Chickering and Garrison Cl 987). The adapted survey used in this study consists of 52 items. Forty items elicit student 
perceptions of the specific outcomes, educational practices, and use of technology. Using a 5-point Likert scale, 
respondents are d irected to indicate to what extent they disagree or agree on items, when comparing the web-based 
course to a course that primarily uses face-to-face communication. Ten items obtain demographic data about the 
students and their educational experiences. Two open-ended questions ask students what they found to be best 
about the course or what needs to be improved. Content and construct validity of the items for the instrument was 
established prior to initiating the study. The reliability of the survey instrument was established from the sample of 
students participating in this study. Chronbach’s alpha for the total instrument was .85. 



Gathering Performance Data 

Data for the pilot stucjy were gathered from 219 students enrolled in courses offered primarily on the World Wide Web 
(www) at three schools of nursing during the fall semester 1999. Descriptive and inferential statistics (correlation, 
ANOVA, and t-test) were used to analyze the data. Content analysis was used to analyze the data from the two open- 
ended questions. For the purposes of this study, the mean for each indicator is reported as the benchmark. Findings 
are reported on the aggregated data across the three schools. 



Results 

The educational practices and outcomes enabled by the technology are revealed by the results of the study on each 
of the performance indicators-outcomes, educational practices, and use of technology. Findings are reported on the 
aggregated data across the three schools participating in the study. 



Demographics 

Ninety percent of the 21 9 students who participated in the study were females. Twenty-four percent of the students 
were age 20-29; 34 percent were age 30-39; 34 percent were age 40-49, and seven percent were over 50 years of 
age. Ninety-three percent of the students identified their race as white. It was determined that approximately 80 
percent of the students had taken between two and six other web courses. Fifty-six percent of the students lived less 
than 30 miles from the campus, and 21 percent lived more than 100 miles from the campus. 



Outcomes 

Intended outcomes measured by this study were accessibility, convenience, sense of connectedness, socialization 
to the profession, and computer proficiency. Survey respondents indicated thatthe courses were definitely accessible 
and convenient (M=3.7, SD=.79). They also believed that the course provided adequate socialization (M=3.5, 
SD=1.01) into their professional role; however, they did perceive a sense of isolation or lack of connectedness (M=3.7, 
SD=.79) with faculty and peers. In general survey students were satisfied CM=3.7, SD=.79) with the web courses and 
indicated that they would continue to take additional web courses, if offered. Although some students perceived that 
they struggled with computer proficiency and that this interfered with their learning, ratings of their abilities to use 
the technology improved significantly from the beginning to the end of the course 0=95.7, p=<.01). 



Educational Practices 




O Time on task. Students in this study reported spending 6-10 hours per week participating in coursework. 
Students neither agreed nor disagreed (M=3.17, SD=1.14) that they spent more time studying in the web 
course than they did in a course offered on campus. 

O Active learning. Active learning requires the student to be a participant in the learning process. Previous 
research indicated that active learning contributes to positive course outcomes and satisfaction. In this study, 
the benchmark indicated that students perceived that they were actively involved in the learning process 
CM=3.3, SD=.84). 
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0 Feedback. Feedback about processes and progress in the course is essential to attaining outcomes. Students 
in this study perceived that they were, in fact, receiving prompt feedback in their web courses [M=3.7, 
SD=1.01). 

0 Interaction with peers and faculty. Interactions with peers and faculty are essential components of the 
learning process. Students in this study were somewhat less likely to interact with peers in the web courses 
than in a traditional classroom CM=2.7, SD=1 .03). Students also were somewhat less likely to interact with the 
faculty when compared to interacting in a course that uses face-to-face discussion fM=2.3, SD=1.06). 



Use of Technology 

Technology [hardware and software) used to offer web courses must be reliable, accessible, and support productive 
use of time. In this study, students disagreed that the technology contributed to good use of their time (M=2.6, 
SD=1.07) and provided a reliable infrastructure [M=2.2, SD=.77). There were geographical differences in students’ 
perception of the utility and effectiveness of the technology. For example, students who lived farther from the campus 
(>51 miles) found that the technology was less able to support productive use of time [F=4.1 2, df=4,21 3, p=<.01) and 
that the infrastructure was less reliable [F=3.0, df=4,2l2, p=<.02). 



Implications and Applications 

The results of this study across the three schools provide useful and relevant data about web-based education. The 
individual school and aggregated data allow for the comparison to our framework of “best practices.” The results can 
be used by individual faculty to strengthen or improve their teaching-learning strategies and by the instructional 
design person to foster the use of these “best practices” in teaching and learning. In addition, administrators can 
examine data in terms of technology support for both students and faculty. All can query the data for the attainment 
of outcomes of a particular course or program. 
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Building Blocks for the 
Adult Learning Focused Institution 



Thomas A. Flint 
Ruth Frey 



Adult Learners: The New Undergraduate Majority 

The stereotyped image of the college student as one who is 18-23 years old in residential, full-time study is being 
challenged by a new reality. The U.S. economy is now information-driven, and a college degree has become an 
increasingly important credential in the marketplace, both for new entrants into the labor force and those already 
employed. Working adults who want to succeed in the present economic climate are pursuing a college education 
in increasing numbers, and they are creating a new majority among undergraduates at college campuses across the 
country. 

Adult students are loosely identified with a larger group characterized as nontraditional. While definitions vary, the 
National Center for Education Statistics (NCES) has come up with seven characteristics that typically define 
nontraditional students. According to the NCES, adult students often: 

o Have delayed enrollment into postsecondary education 
o Attend part-time 

o Are financially independent of parents 

o Work full-time while enrolled 

o Have dependents other than a spouse 
o Are a single parent 
o Lack a standard high school diploma. 

By using one or a combination of these criteria, NCES estimates that more than 60 percent of students in U.S. higher 
education can be characterized as nontraditional. Using the simpler and more common criterion of age to define adult 
learner, we know that some 43 percent [or 14 million) of students in U.S. higher education are 25 or older. And, 
astonishingly, an estimated 65 percent increase in enrollments of students 35 years of age and older, from 1 .7 million 
to 2.9 million, occurred between 1985 and 1996 [NCES, 1996). 



The Unique Educational Needs of Adults 

Many colleges and universities have struggled to adapt to this changing student marketplace, often finding 
themselves burdened by traditions and practices that prove ill-suited for adults. Unlike the returning veterans of World 
War II who went to college under the Gl Bill, today’s adult learners are unwilling and unable to emulate traditional- 
aged students either inside or outside the classroom. Adult students have unique needs, especially if they are 
employed. Among others, these needs include: 

o Different kinds of information about their educational options 
o Institutional flexibility in curricular and support services 
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o Academic and motivational advising supportive of their life and career goals 
o Recognition of experience and work-based learning already obtained. 

These needs reflect how the experience, knowledge, skills, and attitudes of adult learners are different from the 
traditional-aged student. 



Educational Principles That Work for Adults Who Work 

Without good models of effective practice for serving adult learners, colleges and universities will continue to struggle. 
CAEL (Council for Adult and Experiential Learning) has addressed this issue through an initiative designed to assist 
colleges and universities with improving learning opportunities for working adults. 

CAEL partnered with APQC (American Productivity and Quality Center) to conduct a benchmarking study of six highly 
adult-learning-focused colleges and universities. This was a first step toward identifying and disseminating new 
models of how higher education institutions can provide the best possible educational experience for adult students 
(Flint & Associates, 1999). 



Principles of Effectiveness for Serving Adult Learners 

CAEL then transformed the benchmarking study findings into principles of effective practice that have been further 
tested within focus groups of adult learners, educators, employers, union representatives, policymakers, and others 
who are interested in adult learning. These Principles of Effectiveness for Serving Adult Learners describe processes 
and approaches to be adopted by colleges seeking to improve access by and quality for adult students. However, in 
order to ensure flexibility and innovation by institutions, the Principles do not prescribe particular practices or policies. 
Rather, they are meant to serve as a framework for assessing institutional commitment to and capacity for meeting 
the needs of adults, and also to form the backbone of what CAEL calls the Adult Learning Focused Institution (ALFI). 

The eight Principles are: 

1 . Outreach. The institution conducts its outreach to adult learners by overcoming barriers in time, place, and 
tradition in order to create lifelong access to educational opportunities. 

2. Life and career planning.The institution addresses adult learners’ life and careergoals before or atthe onset 
of enrollment in order to assess and align its capacities to help learners reach their goals. 

3. Financing. The institution promotes choice using an array of payment options for adult learners in order to 
expand equity and financial flexibility. 

4. Assessment of learning outcomes. The institution defines and assesses the knowledge, skills, and 
competencies acquired by adult learners both from the curriculum and from life/work experience in order to 
assign credit and confer degrees with rigor. 

5. Teaching-learning process. The institution’s faculty uses multiple methods of instruction for adult learners, 
including experiential and problem-based methods, in order to connect curricular concepts to useful 
knowledge and skills. 

6. Student support systems. The institution assists adult learners using comprehensive academic and student 
support systems in order to enhance students’ capacities to become self-directed lifelong learners. 

7. Technology. The institution uses information technology to provide relevant and timely information and to 
enhance the learning experience. 

8. Strategic partnerships. The institution engages in strategic relationships, partnerships, and collaborations 
with employers and other organizations in order to develop and improve educational opportunities for adult 
learners. 

These statements are further described. and expanded on in CAEL’s executive summary, Serving Adult Learners in 
Higher Education: Principles of Effectiveness (CAEL, August 2000). Colleges and universities have incorporated the 
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Principles into their self-assessment processes. CAEL is also working with several state higher education governing 
boards to design ways for state institutions to better respond to the education needs of adults and business and 
industry. 

At the time of publication of this briefing paper, CAEL is designing an Adult Learning Focused Institution Workbook, 
a self-improvement guide based on the Principles. The focus of CAEL’s presentation at the North Central Association 
Annual Meeting will be to introduce of CAEL’s workbook and to engage attendees in assessing current practices and 
considering new ways of helping adults achieve a college education. 



Summary 

CAEL undertook this project with the aim of fostering improvement among institutions of higher education in an effort 
to assure adult learners an accessible and effective education. It is our belief that the adoption of the Principles will 
result in positive, concrete, and visible changes for our nation’s adult learners. 
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Assessing Faculty Quality 
in Adult Programs: 

Best Practices for Faculty Hiring, 
Orientation, and Evaluation 



Mark A. Smith 
Robert Hamill 



A comprehensive orientation program for adjunct and part-time faculty has a two-fold purpose: to provide a 
connection between these individuals and the institution, and to prepare them to succeed in the classroom. As 
colleges and universities change to meet current challenges, part-time and adjunct faculty are bearing more of the 
teaching load. Successful orientation of these key faculty members, therefore, is an important variable affecting 
institutional effectiveness. 

The diverse and often conflicting challenges facing institutions of higher education are the focus of intense discussion 
within the higher education community. These challenges include rapid expansion at some institutions, heightened 
competition for most, budgetary constraints for others, and the introduction of new models of learning that affect all 
institutions. All of these challenges are causing a paradigmatic shift with far-reaching impact. One of the areas of 
institutional life on which most of these challenges eventually focus is the makeup and role of the faculty. Faculty 
members are being asked to envision their role differently. Perhaps more significantly, institutions are rethinking 
accepted notions of the composition of faculty and the place faculty occupy. Institutions are hiring new “kinds” of 
faculty members, offering them different relationships, and deploying them in new ways. Thompson (1995) 
summarizes the trend in the following way: “Like many businesses across the country, institutions of higher education 
have been increasing the use of part-time and/or temporary staff. Nationally, adjuncts teach between 30°/o-50°/o of 
all credit courses and between 95 Q /o-100°/oof noncredit courses.” Leatherman (1997) believes that adjuncts account 
for 64 percent of the faculty members in two-year colleges. 

As is to be expected, such changes bring confusion, anxiety, and conflict. It is not the purpose of this paper to offer 
judgment about these changes. Rather, this paper will make two points. First, the reality is that adjunct and part-time 
faculty are doing more and more of the front-line instructional work in higher education. Second, in order to prepare 
them to do this well, institutions must offer adjunct and part-time faculty a comprehensive program of orientation 
tailored to their unique needs. 

In the midst of this change, some constants remain. Faculty members need to connect with their institution, to 
understand its mission and values, and to be prepared to meet the needs of its students. In this context, colleges can 
no longer avoid the responsibility to provide a comprehensive orientation program for their part-time and adjunct 
faculty members. Institutions must understand that the quality of this orientation and initial development is a key 
determinant of their institutional effectiveness. John Scott (1 997) argues that, “[b]y supporting and fully integrating 
adjuncts, an institution’s quality of instruction, collegiality, and communication increases." 

One of the simplest and most important steps an institution can take to lay the groundwork for quality learning 
experiences in its classrooms is to ensure that every faculty member participates in comprehensive and ongoing 
orientation and professional development programs. The need for a comprehensive faculty orientation program 
increases proportionally with the level of a college’s reliance on adjuncts and part-time faculty. When colleges use 
adjuncts sparingly to fill holes in the teaching schedule, the overall quality of instruction in the institution will likely 
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not be greatly affected if those adjuncts are not well-oriented to the institution and their role within it. When colleges 
rely-whether by choice or by default-on adjuncts and part-timers Oncluding teaching assistants, one might add) to 
carry a substantial portion of the teaching load, the need for such an orientation increases exponentially. 

Adjunct and part-time faculty do not need to be oriented because they are not full-time. They need to be oriented 
because all teaching faculty need to understand the institutional context in which they work, the curriculum they will 
be expected to cover, and the students with whom they will be working. 

The relationship of an institution with its adjunct and part-time faculty is obviously different from the relationship with 
its full-time faculty. When a full-time faculty member joins an institution, s/he joins a peer group of other faculty 
members who collectively serve the learning needs of their students and govern the academic activity of the 
institution. Adjunct and part-time faculty are often relegated to the fringes of the institution, doing work for which 
they are paid, but not becoming part of the institution’s academic community. Therefore, the goal of the faculty 
orientation program should be to help part-timers and adjuncts connect to the institution. 



What Issues Must Orientation Programs Address? 

At the heart of the comprehensive orientation plan must be a process whereby adjunct and part-time faculty come 
to own the purpose, mission, and values of the institution in which they work. These are the foundational elements 
that connect faculty to and help them understand the institution. 



Foundations of the Orientation Program 

First, faculty must understand the purpose of the institution in which they work. The orientation program must give 
faculty the chance to obtain ownership of the purpose of the institution. 

Second, faculty must understand the mission of the institution in which they work. Faculty must understand the unique 
mission of their institution because it has direct bearing on the expectations brought to bear on them by colleagues, 
students, and administrators. 

Third, faculty should be able to share the core values that shape the ethos of an institution. A faculty member who 
does not know and understand the core values of his or her institution is at a disadvantage. 



Content of the Orientation Program 

The practical goal of a comprehensive orientation program is to ensure that adjunct and part-time faculty are prepared 
to offer students the very best possible learning experiences. An effective orientation program gets the relationship 
between faculty and the institution off to a good start. It lays the groundwork for a mutually rewarding relationship 
in which the faculty member helps the institution accomplish its mission while at the same time finding personal 
satisfaction in doing so. 

To this end, building on the foundations of purpose, mission, and core values, an effective plan of orientation should 
address these three stages of the relationship between faculty and the institution: 

o The pre-hiring stage 
o The connection stage 

o The preparation stage 

□ The Pre-Hiring Stage 

The process whereby adjunct and part-time faculty are hired has been notoriously haphazard. All too often it has 
followed a scenario something like this: A quick perusal of the vita [sometimes optional), coupled with a 
perfunctory interrogation, followed by directions to the classroom-“The class meets Monday, Wednesday, and 
Friday at 7:30 a.m. in room 231. Please be there this Monday.” 

This is an invitation to disaster. Even in the most pressurized situations, where a department head must fill a 
teaching assignment atthe very last minute, time can be taken to do better than this. The initial orientation previous 
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to the first assignment must properly set the stage for the relationship that will follow. Here are some guidelines 
to follow for the pre-hiring orientation. 

First, the pre-hiring orientation should articulate the process that will be used forthe selection of adjunct and part- 
time faculty. This process should be used consistently and should not be compromised except in the most extreme 
emergencies. It should provide for the review of the candidate’s vita, an overview of the institution’s documentation, 
a review of the criteria by which candidates will be selected and assigned, and a set of interviews with key 
representatives O.e., department chairs, deans, and full-time faculty]. 

Second, the pre-hiring orientation should articulate the institution’s faculty selection criteria as they apply to 
adjuncts and part-timers. It should help candidates understand how the institution views adjuncts and part-timers, 
and how the criteria for selecting them are to be applied. This orientation can be especially beneficial if it is explicit 
about the strengths that candidates can bring to the institution, such as a wealth of professional experience or 
involvement in unique positions and projects. 

Third, the pre-hiring orientation should give the candidate a quick overview of the institutional context. While this 
would not be the place to cover the foundational or the practical elements in-depth, it should give the candidate 
the chance to self-select in or out of the process before commitments are made on either side. 

Fourth, the pre-hiring orientation should include an unambiguous discussion of the institution’s expectations of 
faculty. The middle of the teaching assignment is not the right time for faculty and department heads to discuss 
differences because expectations were not made explicit from the outset. 

Fifth, the pre-hiring orientation should provide the candidate with an opportunity to ask questions and state 
expectations. Giving candidates the chance to clarify what is going to be required of them, and to make their own 
expectations known, lays the groundwork for a healthier relationship in the future. 

In summary, the goal of the pre-hiring orientation is to allow the institution to consistently identify individuals who 
match the institutional mission and who can contribute true value to its students. A good pre-hiring orientation 
process will give candidates the chance to self-select in or out of the institution based on their knowledge of 
themselves, their commitments, and their strengths and weaknesses. 

□ The “Connection” Stage 

Once a person has been asked to serve as an adjunct or part-time faculty member, the focus of orientation shifts 
from selection to connection. Adjunct and part-time faculty members usually do not have the benefit of the formal 
and informal systems that help full-time faculty connect with the institution; there are no formal interviews with 
the personnel department, no departmental dinners with introductions, no informal invitations to lunch or the 
fitness center, no passing conversations in the hallways, no frank talks about the structure of the institution and 
the relationships between its departments, and no departmental meetings. In short, there will be few opportunities 
for established faculty to pass along the ethos of the community. There will be no opportunity for adjuncts and part- 
timers to gather those all-important tips and cues about the way things get done. 

Two things must certainly be true in this situation. First, adjunct and part-time faculty will serve the institution better 
if they understand its ethos. Second, part-time faculty will not become connected to the institution without a 
purposeful system of orientation. Some institutions have responded to this need [one will be discussed later] by 
creating intensive orientation experiences that bring adjunct and part-time faculty to campus and, in effect, treat 
them as full-time faculty members during their initial exposure to the institution. The initial orientation plan that 
connects the adjunct to the institution, then, should incorporate the following features. 

First, as discussed earlier, it must provide ample opportunity for the faculty member to become familiar with the 
mission, purposes, and core values of the institution. While these will have been communicated in overview fashion 
in the pre-hiring stage, the initial orientation provides an opportunity for in-depth coverage of these foundational 
elements. The faculty member must be given the opportunity to find his or her own points of connections and 
tension with these elements. In doing so, the aim will be to formulate an understanding of the contribution s/he 
will make to the accomplishment of the mission. 

Second, the initial orientation must acquaint the faculty member with the “nuts and bolts” of the policies and 
procedures he or she will be expected to follow. Here is an all-too-frequent scenario from the classroom of an 
adjunct professor. A student quite innocently and appropriately, asks a question about a procedure or policy of 
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the institution. Adjuncts don't wear name tags to differentiate them from the institution's “real" professors. Hence, 
students expect them to be able to give guidance. But because the adjunct was not oriented well, does not know 
the answer, and, more importantly, does not see himself as a part of the university, he dodges the question by 
saying, “I don't know. You'll have to ask the university." When this happens, everyone in the situation loses. Both 
the adjunct and the institution lose credibility. The student loses faith. It becomes harder for the adjunct to 
accomplish the mission for which s/he was hired. This is why adjunct faculty need to understand the policies and 
procedures of the institution— not to spare the registrar's or the department chair's office headaches, but to serve 
students better and to preserve credibility. Ultimately, the goal of this part of the initial orientation is not to master 
policies, but for adjuncts and part-timers to be able to act as part of the university community. 

Third, the initial orientation should allow faculty to form departmental relationships. Colleges and universities are 
made up of people who accomplish theirwork by means of a diverse set of relationships. The value of the education 
offered within the context of a healthy academic department is not determined simply by the quality of the discrete 
classroom experiences to which a student is exposed. Instead, much of the value comes from the comprehensive 
experience of being exposed to an academic team working together. The whole makes a greater impact on the 
student than the sum of its parts. In a healthy academic department faculty colleagues referto one another's work, 
publicly agree and disagree with one another, bolster one another's arguments, and contribute to one another's 
work by critical interaction. By doing this, faculty challenge students to discover their own points of view. A part- 
time or adjunct faculty member who has not been invited into this community, nor given the chance to form the 
relationships it requires, is at a distinct disadvantage. But even more importantly, the students who sit in these 
faculty members' courses are disadvantaged. Therefore, the orientation that connects the adjunct to the university 
must provide him or her with the opportunity to form these departmental relationships. 

Fourth, the initial orientation should provide opportunities forfacultyto identify and link up with mentors. New full- 
time faculty often select mentors informally. For example, an established professor may serve as a young faculty 
member's informal guide and champion through the tenure process. This role is even more important for adjuncts 
and part-timers because most of them will have their primary professional relationships elsewhere. They need 
someone to whom they can turn for answers when problems arise, when they need new ideas, or for growth within 
the discipline. 

Fifth, the initial orientation must establish a reliable and clearly understood means of communication between the 
faculty member and the leader for the department. Adjuncts and part-timers can become “lone rangers,” who 
become adept at solving their own problems, answering their own questions, and guiding students according to 
their own lights. The orientation program should give them clear communication links with those to whom they 
can turn when answers are needed. 

□ The “Preparation” Stage 

The preparation of most academics focuses almost solely on the content of their disciplines. When a new faculty 
member walks into the classroom for the very first time, almost nothing has been done to prepare that individual 
to facilitate learning. For full-time faculty, this hole is filled by trial and error within the classroom and by informal 
collegial sharing within the department. Neither of these two resources is available to the same degree for adjunct 
and part-time faculty. They teach less than full-time faculty and take longer to build up the volume of instructional 
experience of a full-time faculty member. As has been pointed out earlier, they are not able to draw from the same 
informal collegial relationships as a full-time faculty member. For the most part, they are on their own. For these 
reasons, a portion of their program of orientation must give them at least the basic instructional tools they will need 
in the classroom. Following are some of the key areas that this orientation should include. 

First, new faculty should be able to prepare and use a course syllabus effectively. Some departments provide 
adjunct and part-time faculty with prepared syllabi. This can bean important tool for ensuring the quality of course 
content. Even so, adjuncts and part-timers will be more effective if they understand the function of a syllabus, know 
what their institution requires the syllabus to contain, and anticipate students' questions about the syllabus. 

Second, new faculty should be conversant with the basic concepts of student evaluation and should have a 
thorough grasp of their institution’s grading criteria and grading scales. No other area of classroom work will place 
more pressure on adjunct and part-time faculty than this one. The more tenuous the faculty member’s grasp of 
the institution’s policies, practices, and commitments, the more susceptible s/he will be to the pressure students 
bring to bear on the grading process. 

Third, new faculty should be able to develop basic lesson plans for their classroom sessions. Unfortunately, this 
is a skill most college professors learn by default rather than by design. Students have come to expect something 
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more than a steady pattern of lectures. The days are gone when faculty members can succeed relying on single- 
skill instructional strategies. But neither will they succeed with a disjointed and chaotic jumble of “learning 
activities.” Adjunct and part-time faculty should know how to prepare a coherent plan that focuses the content 
and guides the events of each class session. 

Fourth, new faculty should be forearmed with basic techniques to overcome initial anxiety and to avoid “rookie” 
mistakes. Anxiety can be a hurdle for all newfaculty, and an especially formidable one for adjuncts and part-timers. 
More than one adjunct who could have become an excellent facilitator of learning gave up after one or two 
assignments because of the inability to confidently face a class full of students. Newfaculty will be better prepared 
if they can use such basic techniques as these: 

o Establishing rapport with students before a class begins 

o Being clear with the class about one’s abilities and one’s role as a facilitator of learning, indicating the 
strengths one brings as a resource for students (without shading into braggadocio) 

o Being willing to admit ignorance when it exists 

o Being clear and firm about course expectations without being inflexible 

Fifth, new faculty will benefit from an orientation to the theoretical foundations of their work. Some discussion of 
the differences between pedagogy and adragogy can generate new insights about learning. 

Sixth, new faculty should understand scholarship and professional development expectations. The orientation 
must include a discussion of the institution’s expectations of adjunct and part-time faculty as they relate to 
scholarship and professional development. 

Indiana Wesleyan University (IWU) has dealt with the challenge of preparing adjunct and part-time faculty members 
by developing an extensive selection and orientation program. This program is built on the conviction that the 
corporate leaders it recruits have the appropriate academic credentials to teach, but that they will not succeed as 
facilitators in its adult degree programs without careful preparation. To provide them with this preparation, IWU has 
focused the orientation process on four key areas: 

1. Understanding the mission and purposes of the institution 

2. Understanding IWU policies and practices 

3. Understanding the characteristics and needs of nontraditional students 

4. Understanding both pedagogical and andragogical approaches to the facilitation of learning 

In order to become familiar with the IWU mission, all new adjunct and part-time faculty are required to attend two 
four-hour orientation sessions that emphasize the important distinctives of the university. During the first session, the 
institutional mission statement is reviewed, and the core values of the university are discussed. The session also 
includes an explanation of the learning outcomes that the university hopes each student will achieve. New faculty 
are challenged to embrace the fundamental goal of providing high-quality instruction for every student in every class 
by creating a challenging academic framework for the knowledge base, and by applying this knowledge base to the 
real world. Once this foundation has been laid, faculty members are able to progress to the more practical issues of 
institutional policies and practices. 

Faculty members are introduced to the policies and practices of the university through a carefully designed set of 
exercises that present the policy in question, acquaint the new faculty with the potential problems that might arise 
around that policy, and present possible solutions to those problems. For example, a role play exercise centered on 
a late-arriving student focuses attention on the attendance policy, highlights the problems that experience shows 
occur among these classes when students habitually arrive late, and gives the opportunity to practice ways to defuse 
the situation. An interesting sidebar discussion can reverse the situation and focus on the impact on the class when 
a facilitator arrives late. Another activity focuses on the correct completion of a grade sheet for the records office (a 
simple but surprisingly problematic activity). The evening proceeds with practice and drill types of situations that aim 
to make institutional policies come to life through situations that experience has shown will occur in their teaching 
experience. At IWU, this segment of the orientation is geared to get the attention of the faculty, but also to reinforce 
the importance of following policies that have grown out of hard experience. Faculty members are presented with the 
IWU Faculty Handbook that has been designed explicitly for its part-time and adjunct faculty. 
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During the second session, new faculty explore the characteristics and needs of adult learners and match these needs 
with appropriate teaching methodologies. This session begins with an in-depth exploration of the comparison between 
teaching today’s adult learners and the practices of the past. IWU believes that new faculty must be introduced to this 
approach because most of them will not have seen newer methodologies modeled in their own education. 

Faculty are also introduced to effective pedagogical and andragogical practices. This is particularly important given the 
fact that most of the adjunct candidates are practitioners. These individuals are highly skilled leaders with a wealth of 
valuable experience to bring to the classroom. All of them possess the requisite academic credentials, many from 
benchmark institutions. But almost none of them have been trained in effective facilitation practices. One’s grasp of the 
knowledge base may be outstanding, but in order to effectively facilitate learning, one must understand the different 
approaches to teaching. The specific topics of this session include, “Best Teaching Practices,” “Grading— An Extension 
of the Teaching Process,” “Teaching the Visual Learner,” “How to Build a Syllabus,” and “Preparing the Lesson Plan.” 

The coordinator of the IWU faculty orientation program has indicated that the single most effective change in the 
institution’s process was the commitment to require all faculty to attend two full evenings of orientation. Internal 
assessment processes have shown a decline in negative student feedback about the quality of their classes since the 
second night of orientation was required. Institutions that spend time on faculty preparation reap rewards. 

IWU supplements its orientation program with a mentor system and professional development requirements for each 
faculty member. These continue to enhance the process of student learning at the university. Through the 
establishment of faculty teaching goals, implementation of a faculty observation system that completes more than 
200 peer and administrative class visits each year, the requirement of a faculty growth and development plan for all 
full- and part-time faculty, and the orientation process, IWU is addressing the concerns raised by many opponents 
of part-time and adjunct faculty. 

In summary, effective orientation programs exemplify best practices in the following ways: 

1. Faculty handbook for full-time and part-time faculty 

2. Faculty procedures and services (phone numbers, contact points, etc.) 

3. Orientation sessions (at least one or two four-hour sessions, up to an intensive four-day session) 

4. Emphasis on the mission, purpose, core values, history, and distinctives of the institution 

5. Focus on effective teaching practices 

6. Video and technology training 

7. Incentives for attendance at orientation sessions 

8. Professional development course offered for credit 

9. Required attendance at orientation sessions before teaching 

These best practices show that an institutional commitment of people and dollars is required to effectively orient part- 
time and adjunct faculty members. Faculty, for their part, must also commit time to learn the appropriate procedures 
and best practices for becoming an effective member of an institution’s faculty. 
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Articulating the Roles of 
Off-Campus Faculty to Promote Best 
Practices in Adult Higher Education 



Lana Ivanitskaya 
Megan Goodwin 



Introduction 

How can a university guide, support, and develop its off-campus faculty, particularly those who teach in remote 
locations? How can their essential duties and responsibilities be defined? Which teaching principles and best 
practices can be suggested to instructors of nontraditional students? How can student learning be maximized in 
compressed format courses offered outside of the university’s main campus? Many higher education institutions are 
challenged to effectively integrate their “invisible faculty” into the university community while also providing sufficient 
guidance with the unique aspects of extended degree programs, such as a compressed course format and diverse 
backgrounds of nontraditional learners [Gappa & Leslie, 1993; Greive & Worden, 2000). 

Competition for highly qualified off-campus instructors is keen, particularly given the rigors of part-time teaching in 
compressed adult learning formats. As programs growgeographically distant from campus, higher-level administra- 
tors grow increasingly isolated from the issues defining the faculty role. Faculty and students also continue to struggle 
to meet shifting mutual and institutional expectations, frequently without the benefit of specific guidelines to modify 
performance. The College of Extended Learning [CEL) at Central Michigan University [CMU) is effectively addressing 
issues common to all academic providers, particularly the large percentage of those with off-campus programs: the 
challenge of recruiting, approving, and developing the highest quality faculty. The steps CMU is taking to address 
these challenging issues are outlined in this paper. 



Phase I: Defining the Needs 

A recent initiative by the College of Extended Learning at CMU supported an institutional planning effort to 
incorporate the germane issues common to the university’s faculty into CMU’s off-campus strategic plan. More than 
80 faculty members, academic department chairs, and administrators took part in 1 1 focus groups discussing how 
CMU can address the needs and concerns of its off-campus faculty. The focus group results led to appointment of 
an empowered task force and delivered an action-based college priority: the communication and articulation of the 
roles of off-campus faculty. The latest product derived from this college imperative is a seminal document articulating 
the roles of off-campus faculty. 

Discussion yielded 649 comments in response to the following questions: 

o What are our burning issues and institutional priorities relative to off-campus faculty? 
o How can we better resolve faculty problems and address their concerns? 

o What does leadership need to know in order to include faculty issues in organizational decision making? 

Researchers categorized participants’ comments by content and deduced 1 1 topics. Among these, faculty recruit- 
ment and approval and academic expectations were of greatest interest to participants and accounted for 42 percent 
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of all comments. CMU’s Academic Advisory Council, comprised of extended learning administrators, academic 
program directors, and faculty reviewed these comments and set priorities for addressing the most pressing needs. 
The top priority was defined as follows: 

Articulation and communication of teaching/learning expectations 

1. Articulate CEL’s academic standards and expectations for faculty 

2. Enhance the faculty handbook to address strategies forteaching adult learners in compressed course formats 

3. Increase awareness of academic expectations among CEL instructors 

4. Take a public stand on quality by establishing clear academic expectations. 

Additional issues that have garnered the attention of the Academic Advisory Council include: 

o Improved systems for the recruitment, selection, and approval of faculty 
o Identification of faculty development needs; prioritization of necessary support systems 
o Articulation of criteria for the evaluation of faculty performance and faculty recognition. 

These issues are essential to a comprehensive, high-quality faculty system. They are being addressed using a process 
similar to that described in this paper. However, space limitations do not allow a complete description of these 
systems. 



Phase II: Preparing to Maximize Adult Learning 

Leadership worked with a newly appointed Faculty Project Task Force, consisting of Academic Advisory Council 
members, to design a document that articulated CEL’s measurable expectations of its faculty. Specifically, the Task 
Force worked toward clarification of the faculty role in the academic process and establishment of uniform 
expectations. 

While the Task Force drew relevant material from CEL’s faculty contract and handbook, researchers from the College 
of Extended Learning’s Center for Research on Adult Learning (CRAL) completed a thorough review of the literature 
on adult learning and compressed format teaching. These data were compiled into a comprehensive list of faculty 
responsibilities and teaching expectations drafted in measurable, behavioral terms. 

The resulting document, "Maximizing Learning: CEL’s Expectations for the Faculty Role and Definition of Faculty 
Responsibilities,” listed six categories of responsibility required of all CEL faculty: 

1. Plan teaching sessions 

2. Integrate principles for effective adult learning in flexible delivery format instruction 

3. Assess student learning using reliable and valid measures 

4. Monitor and evaluate teaching 

5. Prioritize professional development 

6. Follow CMU/CEL procedures. 

In addition, eight principles of effective teaching empirically linked to improved learning outcomes for adults in 
compressed course formats were identified: 

1. Understand and respect individual differences 

2. Set expectations and establish purpose 

3. Application 

4. Variety 
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5. Maximize and optimize learners' “time on task" 

6. Communication and cooperation 

7. Feedback 

8. Encourage metacognitive learning. 

Samples of required and recommended faculty behaviors are described in Table 1. 

Table 1 : Faculty Responsibilities and Teaching / Learning Practices 

Required: Assess student learning using reliable and valid measures 

o At the start of the course, explain the criteria for evaluating each graded 
course requirement and the weight of each requirement in determining the 
student's final grade 

o Require multiple class assignments, including pre-course assignments 

o Design multiple assessments of learner performance that link to the course 
goals and objectives 

Principle 5: Maximize and Optimize Learners' * Time on Task" 

o Guide students to set challenging goals for their learning 

o Communicate the importance of frequent self-paced study sessions, 
along with the minimum amount of time students should spend preparing for 
class 

o Establish and communicate systematic learning milestones 



Note. A complete listing of faculty requirements, principles, and best practices is available 
through the Center for Research on Adult Learning at CMU, http://www.cel.cmich.edu/cral/ 



As a final step, the Task Force distributed the draft, along with a request for comments, to engage a university-wide 
discussion of content. The document draft was distributed to academic program councils, department chairs, the 
Chair of the Academic Senate, and focus group participants. This effort led to valuable suggestions that were later 
integrated into the final draft of “Maximizing Learning." 



Phase III: Implementation 

Presently, CEL’s Faculty Project is moving into its next phase: implementation of the “Maximizing Learning" document. 
The goal is to better align CEL’s expectations of its off-campus faculty with its ability to accurately assess and support 
faculty needs. Consequently, members of the Faculty Project Task Force and CRAL are working in concert to create 
a teaching/learning assessment battery focused on best practices in adult learning. This assessment battery will offer 
multiple measures of faculty performance from several key perspectives: self, students, peers, and mentors. It will 
leverage results toward a more individually flexible system of faculty coaching and mentoring. In a related effort, 
“Maximizing Learning” was used to derive a set of academic expectations for CEL’s adult learners, summarized in 
“Maximize Your Learning." 

As noted previously, to drive the utilization of practices for maximizing adult learning, the College of Extended 
Learning is carefully reviewing its processes for faculty recruitment, selection, and development. For example, a state- 
of-the-art Faculty Credentials Evaluation will help select those applicants for an instructor position who have 
demonstrated the following competencies: ability to teach working adults, expert knowledge of course content, 
research and field experience. After a new faculty member receives an approval to teach a CMU class, he or she needs 
to be given appropriate guidance and training. CMU’s faculty mentors provide ongoing faculty support and coaching 
in addition to the regular orientation and training sessions for off-campus faculty. Plans are being made for adopting 
“Maximizing Learning” as a primary resource used by the developers of faculty orientation and training sessions. 
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Conclusions 

“Maximizing Learning” is more than a committee document; it is a conduit for the sharing of academic standards, 
performance objectives, and best practice techniques among off-campus faculty and their on-campus colleagues. 1 
Having earned the support of CMU’s President, College Dean, and Academic Program Councils, “Maximizing 
Learning" is now a living document, capable of concentrating finite resources where they can have their greatest 
effect-driving utilization of best teaching practices that directly benefit adult learners. These efforts are only the first 
step toward a broader focus on quality, accountability, and support for an essential component of external degree 
programs-off-campus faculty. 

CMU’s initiative can serve as a working template for the assessment, analysis, and quality enhancement of existing 
faculty systems. The process and practice behind CMU’s initiative provides an opportunity to effect similar quality 
improvements at other institutions. 
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Note 
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Mentoring the Adult Learner 
in Distance Education 



Hilda R. Glazer 
David S. Stein 



Retention of adult learners in the distance doctoral, teaching, and learning environment is an issue of concern to 
faculty and administrators alike. The improvement of retention rates is critical during this time of decreasing 
enrollment in both campus and virtual graduate programs. Retention efforts focus on the beginning students as they 
orient to on-line learning and distance education and adjust to independent learning. 

Using the doctoral programs at Walden University as the model for this discussion, retention has been an area in which 
effort has been focused during the past two years. Incoming students are typically supported by an orientation or start- 
up team and specially selected mentors to guide them through the first year. Often, progress is closely monitored in 
terms of benchmarks to be completed during the beginning quarters. Learners are motivated by the elation of 
beginning the program of study. The advanced students have the support of dissertation committees and are 
motivated by being able to see an end to their program of study. It is the students between these two groups who 
are of particular concern in this presentation: these are the students in academic mid-life. While most students who 
drop out from the Walden doctoral program do so during the first 1 2 months, approximately 20 percent of those who 
drop out will do so during the second year of enrollment. There is a curious increase in the dropout rate for all programs 
during the sixth quarter in the program. 

Academic midlife is a stage between the completion of first-year tasks and the acceptance of a proposal. It is during 
this stage that students are typically more isolated from faculty due to the fact that they tend not to maintain frequent 
contact with the mentor, but rather work with individual faculty members for shorter periods of time. These students 
also tend to have completed residency requirements and as a result have less face-to-face contact with the university 
community. The excitement of an academic program becomes tempered with the requirements of family and job, and 
learners may become bogged down with the press of academic requirements and the search for a dissertation topic. 

Academic and social integration is critical to persistence and degree completion; that is, the student feels part of the 
community (Towles, Ellis, and Spencer, 1993]. Faculty-student contact has been identified as a variable impacting 
persistence in campus-based programs as well as distance education. Most of the research in this area has focused 
on undergraduate education and situations in which students chose one course using a distance format. However, 
the results of these studies can lead to the identification of variables for investigation of student factors in persistence 
in any distance education program. Research results with undergraduates have been inconsistent with regard to 
faculty-initiated contact, though trends indicate that faculty-initiated contact had a greater impact on completion with 
lower classes (Towles, Ellis, and Spencer, 1 993]. In a study by Pugliese (1 994], none of the variables predicted to be 
associated with withdrawal from telecourses were found to predictive: these included locus of control, loneliness, 
social confirmation, social experience, and dyadic communication with faculty and tutors. In looking at systems to 
enhance the learning experience for the distance learner, research by Dillon, Gunawardena, and Parker (1992] 
supported the need for access to student services on-line and the importance of effective interpersonal communi- 
cations between faculty and students. The results of this study suggested that teachers should be encouraged to 
assume a more active role in communications. Consistently, access to faculty, other support systems, and integration 
within the learning community have been identified as critical to success. 

The objective of the present study was to identify from narratives of faculty and students how the student maintains 
a high level of integration in the distance academic community. What level of academic support is desired, and what 
level did the students receive during this period? 
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The Virtual Campus 

One of the debates about the Internet has been whether it leads to more and better social relationships or if people 
become more isolated and cut off from genuine social relationships (Kraut et al., 1 998). While the Kraut et al. study 
did not focus on educational outcomes, it did support the hypothesis that time on the Internet adversely affects social 
involvement and psychological well-being. An early study of German distance education by Kahl and Cropley (1 986) 
identified a set of characteristics of distance learners: they often elect to work with a set plan, set aside special times 
for learning, and set up a learning area for themselves: they prefer learning materials that are clear, explicit, and 
structured. 

The question for the midlife learner whose primary relationship with other learners and with faculty mentors is through 
the Internet is how to enhance the quality of those relationships so that they enhance the educational experience 
rather than adding to isolation and decreased social involvement. 

This presentation explores and describes the strategies developed by faculty and students to provide academic 
support for the midlife student. The researchers used a variety of data collection techniques: online student interviews 
in the form of narratives, on-line faculty discussion over a three-week period, and an online focus group with student 
volunteers. The data were subjected to qualitative analysis techniques to identify patterns, themes, and trends 
regarding the interaction of faculty and students during this critical period. 



A Developmental Perspective 

Peck (cited in Papalia, Olds, and Feldman, 1998) identified four psychological adaptations necessary in middle 
adulthood. They are valuing wisdom rather than valuing physical prowess, socializing as the primary value in 
relationships, emotional flexibility, and mental flexibility. Perhaps the one that is critical to this discussion is socializing 
as a primary value in relationships. Developing and maintaining a relationship with the mentor and with others in the 
graduate school cohort cannot be discounted. The mentoring relationship may hold the key to successful transition 
in graduate school and the successful adjustments to be made not only within the Walden community but also in the 
wider community both during and after the program. 



The developmental literature can provide a perspective for looking at the midlife learner. Education is seen as being 
related to the development of identity in adult women. Petersen (2000) found that, for Caucasian women, education 
was a way to discover competence and to modify or transform established roles. Their goals and motivators appeared 
to revolve around personal emotional gains. For the African-American women in this study, barriers to education and 
opposition were the typical experiences in higher education rather than experiences that enhanced self-esteem. Thus, 
self-esteem and identity development can provide motivators for midlife women if barriers to success can be reduced 
or eliminated. Helson (1993) found in her study of Mills College graduates that, in contrast to previous research, 
women do not become less dependent over time when they have followed the “ feminine social clock.” Rather, it was 
the influence of multiple roles and tasks that women select: that is, it is critical to study subjects, both men and women, 
in life contexts. Thus the context in which graduate study occurs is the critical piece. 



The development of goals is also related to success as a learner. Gollwitzer (1 999) suggests that goal attainment is more 
likely when people frame their good intentions as learning goals rather than performance goals. Success depends on 
being successful at self-regulation; that is, being able to initiate goal-directed behavior and bringing it to a successful 
conclusion. Part of this is control of the environment; “the person prevents the derailing of an ongoing goal pursuit by 
removing the competing temptations from the situation in which the goal pursuit is to occur” (Gollwitzer, 1999, p. 494). 
Successful goal pursuit requires tenacity and flexibility (Gollwitzer, 1999). This is particularly important for the individual 
working independently and often without the visual and real support of others doing the same thing. 

The relationship between the learner and faculty mentor has been seen by Walden as a critical part of the learning 
process. Defining that role and relationship for the midlife student is one of the focuses of this study. What are the 
characteristics of the successful mentoring relationship during this stage of academic life? Walden University works 
to develop opportunities for students and faculty to maintain contact with the academic community. This task is more 
difficult to achieve in the virtual environment. Innovative uses of student listservs and bulletin boards, residency 
options, and academic support services available on-line and by phone are provided. Which ones are desired and 
needed by the midlife student becomes one of the questions addressed in this study. 




Keegan (1998) proposes that the manner in which a distance education institution reintegrates the teaching act with 
the learning act influences learner retention and the quality of academic performance. This theory informs our study 
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of academic midlife learners. Keegan’s theory suggests that the student’s retention is enhanced when academic 
support services are available that integrate the student into the academic community or provide the student with 
the feeling that he or she is a member of the academic community even though the student is at a distance. In distance 
education, learners often do not have access to immediate learner-to-instructor or instructor-to-learner feedback; 
reinforcement may be delayed; and peer and academic support can be lacking. Reintegrating the teaching act with 
the learning act reconstructs the interpersonal relationships that exist in the face-to-face classroom. Keegan 
hypothesizes that the separation of the teaching act and learning act is responsible for a weak integration of the 
student into the scholarly life of the institution. 

This lack of integration may contribute to students’ dropping out of the learning experience. Further, the separation 
of the teaching act and learning act is responsible for a weakness in interpersonal communication, leading to a lack 
of quality in the learning achieved. This study explores how reintegration of the teaching and learning acts can occur 
in a distance learning environment. 



Method 

□ Subjects and the Learning Environment 

The faculty and learners participating in this study are from Walden University. Walden offers doctoral programs 
in two formats. The first is the traditional course format that characterizes the programs in the Division of 
Psychology and all MS programs. Courses are offered in an on-line format and in a format that combines on-line 
instruction and face-to-face learning. The other doctoral programs in Management, Education, and Health and 
Human Services use an independent study format in which a learner works with an individual faculty member to 
develop a learning agreement based on knowledge-area modules. The product of the learning agreement is a 
three-part demonstration of learning over a well-defined area of knowledge. 

The student sample consisted of 36 doctoral students; 1 9 were enrolled in the independent-study (KAM) program 
and 1 7 in the course-based program. Of these, 25 (69 percent) were female and 11 were male. Time in the program 
varied, with a mean time of 23.75 months. All subjects completed consent forms, which they mailed to the 
investigators. 

□ The Interview Questions 

The interviews were conducted on-line. Questions were sent to the list of participants and participants could 
respond to the group or to the investigators privately if they preferred. One question was sent out every two days. 
The questions were; 

1 . What did your mentor or other faculty members (without using names) do, say, or provide that kept you 
feeling part of the Walden community? 

2. How did your mentor or other faculty members keep you progressing in your program? 

3. What did you do to keep yourself focused on your studies, given the demands of your life? 

4. In what ways did you keep yourself part of or identified with the Walden community of scholars? 

As needed, the investigators sent follow-up questions requesting clarification or additional information. 



Thematic Analysis and Code Development 

Thematic analysis is a process for encoding qualitative information using themes generated inductively from the raw 
information. The specific procedure used was developed by Boyatzis (1998). 

learners 

Faculty respond to queries or concerns in a timely manner (24-48 hrs). 
Answered my question promptly; always available when I called. 
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Theme 1 . Responsiveness to 

Definition: 

Indicators: 
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Theme 2: Relationship with learners 

A. Supporting 

Definition: Faculty offer reassurance indicating that one can succeed in the program. 

Indicators: Tells students that “We are in this together"; keeps in regular contact to offer 

support and encouragement; expresses sincere interest in my success. 



B. Academic progress 

Definition: Consistent contact with the learners to ensure that progress in completing 

courses/KAMS is occurring; provides information on learning resources. 

Indicators: Keeps in touch regularly to check on progress; responds in class postings 

(individually); provides advice on sequencing courses. 

C. Respectful 

Definition: Faculty reduces distance in status and treats students as adults. 

Indicators: Faculty have been courteous, interested, cooperative with students; makes me 

feel like a valued and competent part of the community; meets me face to face 
to read a paper. 



Theme 3: Resources 

Definition: Links learners to a community of other learners or faculty. 

Indicators: Initiated a listserv for contact with other students; connected “newbies" with 

experienced students. 



Discussion 

Analysis of the student and faculty narratives indicate that frequent meaningful communication seems to be the key 
to retention. Communication appears to be a metaphor for caring about the individual; the perception that one is cared 
about makes the difference. Being responsive is an important element of this. A second theme is the importance of 
knowledge of the program mechanics and how to work through it and knowing that the mentor is able to help the 
learner through the bureaucracy. It appears that the mentor and learner through the quality of the interaction are 
trying to simulate face-to-face interaction in the virtual environment. The third theme is that now is the time that the 
mentor begins to stress academic values as a way of integrating students into the academic community, as opposed 
to the initial focus on content. 

One theoretical perspective that may account for this is the cognitive-experiential self-theory of Epstein (1993). The 
way in which people approach the world is through an experiential system, a rational system, and a primary process 
system. The experiential system is context-specific, and judgments about actions are based on past experience. The 
rational system brings to bear contextually generated general principles. These decisions are done within a context 
of meeting needs for relatedness, self-esteem, and coherence. Thus, if we decrease the relatedness and increase the 
isolation of the individual, decision making becomes more and more based on previous experience and maybe unable 
to take into account the contextual differences related to distance learning. 
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Preparing Your Organization for AQIP 



Jane A. Bishop 
Dennis Ladwig 
Michael Lanser 



Introduction 

Lakeshore Technical College (LTC), one of Wisconsin’s sixteen technical colleges and part of the Wisconsin Technical 
College System, operates under a shared governance concept with the state and local boards equally responsible for 
setting and administering policies. The college offers associate degree and technical diploma programs, and adult 
and continuing education courses. LTC received its ten-year NCA accreditation in 1992. After the accreditation 
process, a plan for assessing student academic achievement was developed and committees were organized to carry 
it out. Although a plan was developed, efforts to implement it were fragmented and overall unsuccessful. The ability 
to measure outcomes was an im portant factor in determining if LTC was meeting its customers’ needs. Demonstrating 
the ability to measure outcomes was the critical question in determining whether the college was ready to join AQIP. 
This paper discusses the journey Lakeshore Technical College Q-TC) took to prepare for entering the AQIP process. 

Developing a System to Identify and Monitor Results 

The Student Academic Achievement Task Force was formed in 1993. The purpose of the task force was to develop 
a model to measure the accomplishments of students’ goals. The group identified the focus for initial assessment 
efforts. Occupational competency programs were identified as the most critical for data collection and analysis to 
improve educational programs and services. In addition, eleven critical student achievement assessment elements 
were identified including: (1) quality employer satisfaction, (2) quality student satisfaction, C3) quality student support 
services, (4) quality articulation, (5) quality instruction, (6) quality staff, (7) quality environment, C8) quality evaluation 
by Students, (9) quality facilities and equipment, 00) quality career exploration, and Cl 1) quality student life. Three 
items were selected-quality of instruction, quality of staff, and quality of employer satisfaction. After achieving its first 
purpose of developing a model, a second purpose was drafted. The new purpose of the task force vyas to develop a 
road map for use of measurementtechniques for each element in the initial phases of instruction, staff, and employer 
satisfaction with occupational programs. The task force analyzed and prioritized North Central input and identified 
training needs to use systems and processes to implement the model. 

At the same time LTC was developing this model, the Wisconsin Technical College System was developing a model 
for colleges in the system to guide their data collection and assessment efforts. The priority in implementing the model 
was identified to be those activities instrumental in student success. Seventeen Core Indicators of Institutional 
Effectiveness were developed through input received from focus groups, analysis of North Central Association 
guidelines, and federal legislation. The core indicators focused on four key areas of effectiveness-student academic 
achievement, employer satisfaction, organizational quality, and public perception and satisfaction. The seventeen 
core indicators are (1 ) identification of student needs, goals ,and interests; (2) identification of student functional skills 
at entry; C3) course completion; (4) student grades; C5) student satisfaction with courses, programs, and services; 
C6) student retention/withdrawal rates; (7) student completion and graduation rates; C8) student achievement of 
educational goals; ( 9 ) student knowledge and skills at exit; 0 0 ) pass rates/scores of licensure exams; (1 1 ) placement 
rates/employment success; 02) employer satisfaction with graduates' work skills/performance; Cl 3) achievement of 
institutional goals and standards; 04) organizational climate; 05) articulation and linkages with external organiza- 
tions; 06) identification of customer needs and expectations; and (17) public satisfaction. The WTCS model was 
intended to provide a means to measure the performance of the entire state system. 

As part of the creation of the core indicators, a survey was developed to identify what was currently being done at 
the college, who was responsible, and the status of the effort. Additionally, staff were asked about the future direction 
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of assessment in their areas, the timelines, and the people responsible. Plans from two other national colleges were 
reviewed and served as initial benchmarks. Based on the evaluation of the plans and the survey results, an 
implementation plan was developed. The following activities were identified: (1) establish a framework for measuring 
institutional effectiveness, (2) form a steering committee, (3) allocate resources, (4) set priorities for assessment, 
(5) develop assessment measures according to priorities, and (6) establish data/information collection standards. 

In 1996, the Institutional Effectiveness (IE) Steering Committee was formed. The steering committee had 
representation from all areas of the college, and it provided leadership for determining measures for the core 
indicators in the WTCS Institutional Effectiveness Model. A faculty member was appointed chair, which assured 
faculty involvement and buy-in. For the next year, the committee worked on developing definitions and 
measurement parameters for each of the indicators. An annual “IE Core Indicator Status Report” was produced. 
It provided the LTC staff with (1)the seventeen measures, (2) the measures’ definitions, (3) the person(s) 
responsible for gathering/reporting the data, (4) how/when the indicator was measured, (5) how/when the 
information was reported, (6) the data source/report name and statement of the target, (7) how the target was 
determined, (8) the interpretation as to how close LTC was to meeting the target, (9) update on findings for the 
core indicator from the previous year, (10) the current year opportunities for improvement, (11) statewide 
benchmarks/recommendations, and (12) miscellaneous information. 

Accompanying the “IE Core Indicator Status Report” was the “Institutional Effectiveness Assessment Report.” Initiated 
in 1997, this report showed the actual results for each indicator, the comparison results to the previous year, the 
comparisons to the benchmarks, and the findings/conclusions. This report was also published annually and 
distributed to all service areas. 

Measurement using the IE Core Indicators began during the 1 995-96 school year. One indicator was used. In 1 996-97, 
eleven of the seventeen indicators were used as measures. All seventeen indicators have been used since 1997-98 
and continue to be used today. As the measurements are accomplished, the Institutional Effectiveness Core Indicator 
Report Card is updated to reflect the current status. The report card is found on LTC’s intranet and is distributed to 
all staff and the district board. Please refer to Figure 1 for the current LTC Report Card. 



Selling the AQIP/Baldrige Concept to the LTC Executive Committee, the District Board, and 
the Staff 

In the fall of 1999, the chair of the IE Steering Committee proposed to the LTC Executive Committee that the college 
pursue the Malcolm Baldrige/AQIP categories for the next NCA accreditation that was slated for early 2002. Extensive 
study and research on the Malcolm Baldrige categories, the traditional NCA criteria, and the Core Indicators of 
Institutional Effectiveness took place. It was critical for the measures LTC was using to match what was needed for 
the Baldrige/AQIP processes. Please refer to Figure 2 for comparison of the Core Indicators and the traditional NCA 
criteria and to Figure 3 for comparison between the Core Indicators and the AQIP categories. 

In addition, the 1 999-2000 strategic plan had identified the measurement of outcomes as one of its priorities. The core 
indicators of student satisfaction with courses, programs, and services; student retention/withdrawal rates; employer 
satisfaction with graduates’ work skills/performance; and achievement of institutional goals and standards were the 
accountability measures in the strategic plan. 

The initial reaction of the Executive Committee was uncertainty. They were familiar with the traditional approach and 
were not confident that the college had enough data or the systems and processes in place. In order to assist the 
Executive Committee in making the decision to employ the AQIP categories, the Research Department conducted an 
analysis comparing AQIP measures to the college’s institutional effectiveness measures. Although it was not a one- 
to-one relationship, there was a strong correlation between the measures. On the basis of the discussion and 
demonstration of the evidence, the Executive Committee agreed to use the AQIP process of accreditation. 
Presentations contrasting the Baldrige/AQIP criteria and the traditional NCA criteria were delivered to the District 
Board and the entire LTC staff. 



Integrating AQIP into the College's Strategic Planning Process 



In the spring of 2000, the college’s strategic planning process was redesigned, and a continuous improvement 
timeline aligned with the AQIP process was developed. The process consists of assessment, priorities, and strategies. 
c ive-year performance measures for each strategy were developed, and targets were identified. The Core Indicators 
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of Institutional Effectiveness serve as the skeleton for the measures. Discussion in planning meetings now revolves 
around howto measure progress. 



Writing an Application for the Wisconsin Forward Award 

As part of the criteria for AQIP membership, each college needs to obtain external feedback on quality criteria. LTC 
decided to use the Wisconsin Forward Award, Wisconsin's version of the Malcolm Baldrige criteria as its external 
validator. The application was submitted in June 2000. The preliminary materials gathered for assisting the Executive 
Committee in their decision about AQIP served as the basis of the application. The Institutional Effectiveness Core 
Indicators provided most of the results. In October, the college was informed that it had won the Mastery level of the 
Wisconsin Forward Award, indicating that LTC had developed and implemented effective and systematic approaches 
to many of the key requirements. 



Using Feedback for Prioritizing Areas for Improvement 

The “Wisconsin Forward Award Feedback Report" indicated that continued measurement needed to occur in order 
for positive trends to be illustrated over time. The report also indicated that although the college had identified and 
developed key strategic objectives and priorities, it had not developed specific timetables and evaluation methods 
to assess meeting the objectives. Since the heart of what the college does is identified in its strategic plan, continuing 
to monitor/measure progress in order to continuously improve the college and student learning is the next critical 
step in the AQIP process for the college. 



Jane A. Bishop is Vice President of Enrollment Management at Lakeshore Technical College in Cleveland, Wisconsin, 
Dennis Ladwig is President of Lakeshore Technical College in Cleveland, Wisconsin. 

Michael Lanser is Vice President of Education Support at Lakeshore Technical College in Cleveland, Wisconsin. 
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Figure 1. Lakeshore Technical College Institutional Effectiveness Core Indicator Report Card 
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Figure 2. 

Lakeshore Technical College 

Comparison of the Institutional Effectiveness Core Indicators and Traditional NCA Criteria 
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Figure 3. Lakeshore Technical College Comparison of the Institutional Effectiveness Core Indicators with the AQIP Criteria 
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AQIP Quality Improvement Teams: 
How One Worked, Lessons from Others 



The concept that colleges and universities ought to constantly improve and that more than just a few people should 
be involved usually enjoys wide support. The challenge arises more from the “howto ...” than the “What to ...” 

With the insight and the experience from personally building more than 300 continuous process improvement teams 
and forming more then three dozen steering committees, some lessons have been learned. The following principles 
outline some of the basic “how to’s” of one successful way to effectively implement quality improvement. 

Preparation for Quality Improvement 

When setting the stage for quality improvement or continuous process improvement, it is vital that top leadership view 
this as a commitment that is continuous. It is not a flavor-of-the-month educational fad. Therefore, it is important to 
set firmly the foundation of support, including support by the president, vice presidents, deans, and the board. This 
full foundation of support may take some time to develop and usually needs clarification before there is sufficient 
support to assure a great start. 

0 Key points of preparation 

o Decide who is to facilitate the quality improvement training. 

o Meet with president and other champions of quality improvement-long-term commitment needs to be 



o If the board is not included already, gain initial support. 

o Initial training of leadership and a cross-section of administration, faculty, and staff, usually for 
approximately two days. This training includes: 

- Foundation principles 

- Seven-step process improvement model 

- Introduction and use of quality tools 

- Integration with AQIP and NCA. 

Establish a Quality Steering Committee 

The quality steering committee is a horizontal representative slice of the entire institution. It needs to include the 
president and any leadership who will be needed to make decisions and support teams and their recommendations 
for improvement. The steering committee’s role is to guide or steer the process of quality improvement. 



Stanley Jensen 
John Erwin 



"How to . . ." Just as Vital as "What to . . 



gained. 
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The role of the steering committee: 

o Guide the quality improvement process for the entire institution, 
o Guide and support quality teams, 
o Help choose team members. 

o Steer teams into the strategic areas of most need or of most help to the institution, 
o Coordinate improvement theory implementation, 
o Guide strategic planning for the future of quality improvement, 
o Coordinate the AQIP-NCA process. 

Typical members of the steering committee: 
o President 
o Vice presidents 
o Deans 
o Directors 

o Representative of faculty, staff, others areas. 



Structure and Function of Teams 

The use of teams is not the only way to implement quality improvement, but teams are one of the main ways to realize 
not only the constant improvement of processes, but also the involvement of many people in real change and in 
breaking down barriers between divisions. 

Team size is more of a factor than most may think. Smaller teams seem to work more effectively and efficiently. Teams 
of four to six members tie up less time and energy and aid in the ever-difficult scheduling battle. The small team can 
still tap many other minds in the institution without requiring their presence on the team. 

0 The typical structure of teams 

o Four to six members 

o A team leader chosen by the team to help coordinate and organize the team’s efforts; in addition, the 
leader stays in communication with the steering committee 

o A team scribe chosen by the team to keep notes and aid in organizing the team’s efforts 

o The team sponsor, a member of the steering committee whose role is to help the team achieve success 
and to act as the communication link between the steering committee and the team 

The steering committee chooses strategic processes or areas that are in the most need of improvement or are of the 
most importance to the institution. The steering committee then chooses team members who are logical as part of 
the individual teams. The team is given a general time line and a date to report back to the steering committee. The 
steering committee coordinates the recognition of the teams and their successes. 



Typical Function of Teams 

The team will typically meet regularly. Some meet each week, some every two weeks, and some once a month. The 
key is to have strong and clear facilitation of the team meeting and great training. 

Teams will learn: 
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o 



The philosophy of continuous improvement 
The six principles of continuous improvement 
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o A seven-step approach to continuous improvement 
o About 30 tools of continuous improvement 
o Focus on understanding students' and other stakeholders’ needs 
o How to measure effectiveness 

o How to effectively value team members and others throughout the organization 
o How to improve processes and how to implement those ideas in cost-effective ways 
o How to optimize the whole institution, not just parts of the institution. 



Measurement and Communication of Outcomes 

If great things happen as a result the continuous process improvement effort, but no one attributes them to the effort, 
the building of momentum will be blunted. Clear measures of results will be needed for a number of reasons: 

o To clearly measure the effectiveness of the theory for improvement 
o To use for charting the course for further improvements 

o To clearly identify gains that can be celebrated and recognized throughout the whole institution 
o To connect to the implementation of the AQIP model and NCA accreditation 
o To gain even more support for continuous process improvement. 

Communication of the outcomes and the specific measures is not to be taken for granted. They need to be 
communicated in nearly every way possible. Educators and students alike are constantly bombarded by thousand of 
messages each week. It is easy to miss even important communications. So something much more than a memo or 
an e-message is needed. This communication needs to become a part of the very fabric of the institution and the 
board. 



Summary 

The concepts of continuous process improvement or quality are great, and they work effectively in education. But the 
implementation of the concept is not easy and should be approached with care and insight. It is hoped that the 
experiences at Cincinnati State and three other colleges that have aided in the preparation of this approach will lend 
some ideas and guidance to other colleges and universities as they embark on the uses of quality and the AQIP/NCA 
model for institutional improvement. 



Stan Jensen is President and founder of Leadership Enterprises Inc., in Bondurant and Des Moines, Iowa. 
John Erwin is Academic Vice President of Cincinnati State Technical and Community College in Ohio. 
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From Rhetoric to Results: 

One Department's Initial Experiences 

with AQIP 



William E. Roweton 



The North Central Association’s (NCA) Academic Quality Improvement Project [AQIP, 2000) revitalizes postsecondary 
institutional program assessments (Ldpez, 2000). Empirically, AQIP assesses patterns in postsecondary student 
learning with contemporary data-analytic and design strategies. At least historically, AQIP demands much from 
postsecondary educators. 

Through AQIP, a college’s initial NCA program assessment plan transforms itself, for example, 0) by utilizing direct 
measures of student learning; (2) by incorporating assessment results into instructional decision-making; and (3) by 
developing psychometrically. In this program, challenges of evaluating student learning and of implementing AQIP in 
a small, rural Midwestern college will be explored. 



Implementing AQIP 

Effective program assessments of student learning, especially their quantitative designs and statistical treatments, 
rarely walk off-the-shelf. Through trial and error, productive strategies cobble themselves together creatively from 
professional experience, skill, and persistence. Moreover, with AQIP, data drive instructional decision-making; 
assessments plans respond to measures of their own effectiveness; and student learning is assessed directly. 



Utilizing Direct Measures of Student Learning 

The focus of AQIP... is student learning. (AQIP, 2000, p. 4) 

Direct measures of student learning refer to assessments of what students do. With some postsecondary program 
assessments, for example, measures of student performance occu r before and after instruction, e.g., basic interrupted 
time series design (Orwin, 1997; Spector, 1981). Student performance, that is, measures of learning, improves 
following instruction. Often, data interpretation amounts to statistically linking key instructional variables, such as 
instructional style, to levels of performance change. Understanding relationships between input and instructional 
variables sometimes yield instructional-planning insights. 

Success with AQIP may depend on productively coordinating (1) measures of student learning and (2) effective 
interpretive tools rendering results useful for instructional planning. The task is rarely simple. 

Student learning reflects numerous input, idiosyncratic, and situational factors, only some of which accommodate 
instructional planning. Determining which variables enjoy both social and statistical significance and enrich 
instructional planning compels some (e.g., Astin, 1993/1991; House, 2000) to champion multivariate approaches to 
program assessment. 

“If reality is complex,” Hubert Blalock (1970, p. 71) observes, "so must be the analysis!” 
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Incorporating Assessment Results 

How do you use information aboutyour ... performance ... to improve your [instructional] approach [to student 
learning]? How does your institution learn to improve continuously from your experience with these processes? 

[AQIP, 2000, p. 10) 

Effective program assessments for AQIP reflect more than valid educational visions shared professionally, or just 
psychometric sophistication [e.g., Braskamp, 1991; Stake, 1967), or merely cognitive and instructional research. 
Program assessments are valued because they can enhance educational planning. 

However, capturing decision data from natural learning environments challenges our design and statistical skills and 
our interpretive imagination. Furthermore, connecting program assessment data to instructional planning decisions 
remains poorly mapped. Nonetheless, strategies for linking assessment results to decisions will be discussed. 



Developing Psychometrically 

...an institution will articulate its goals with measurable precision so that its performance in achieving them can 
be consciously tracked. [AQIP, 2000, p. 4) 

Healthy program assessments grow and prosper, that is, evolve. Productive ones confront, resolve, and, ultimately, 
discount initial empirical queries advantageously. That’s growth. 

When an institution’s assessment plan does not learn from its own experience, refurbish it technically. Criteria for 
evaluating growth in technical aspects of program assessment will be discussed. 



Conclusion 

Together, valued and testable educational hypotheses about human learning, realistic methods of data management, 
sufficiently complex program evaluation designs, and professional motivation promise improved postsecondary 
program assessments of student learning. To do so as AQIP recommends, [1) utilize direct measures of student 
learning, [2) incorporate assessment results into planning, and C3) develop program assessments psychometrically. 
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Charting Our Course to 
Academic Quality Improvement... 
The Western Way 



Jerrilyn Brewer 
Diane Osterhaus Neefe 
Jane Rada 
Lee Rasch 



Introduction 

Western Wisconsin Technical College (Western) has charted a course to academic quality improvement by 
integrating its continuous quality improvement initiatives and its preparation for North Central Association (NCA) 
reaccreditation. Western’s participation in NCA’s Academic Quality Improvement Project (AQIP) has provided unique 
opportunities for the college to enhance its ongoing quality initiative by identifying key areas for improvement. 
Additionally, the AQIP process is enabling the college to use its resources and its energy to focus on improvement 
instead of focusing on preparation for an NCA site visit. This paper will provide practical information for other colleges 
that be considering AQIP as an alternative form of reaccreditation. 

Western’s continuous quality improvement program embraces the core values and concepts set forth in the Malcolm 
Baldrige National Quality Award Program. The college believes that these values and concepts are embedded beliefs 
and behaviors found in high-performing organizations. They include the following: 

o Visionary leadership 
o Customer-driven excellence 
o Organizational and personal learning 

o Valuing employees and partners 

° Agility 

o Focus on the future 
o Managing for innovation 
o Management by fact 
o Public responsibility and citizenship 
o Focus on results and creating value 
o Systems perspective 

These values and concepts are central to Western’s mission, vision, values, beliefs, and strategic priorities. They form 
the framework for the selection of what the college considers to be critical elements that define the Western Way: 
collaboration, communication, caring, and commitment. 
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This paper will describe these critical elements and will provide examples of how they help the college Chart our 
Course to Academic Quality Improvement. Areas to be addressed include: 

o Building strong linkages with other higher education institutions 
o Focusing strategic planning efforts 

o Identifying comparable performance measures and benchmarking 
o Strengthening commitment to students, key stakeholders, and staff 

o Enhancing a key college value of working together in a healthy environment where creativity, humor, and fun 
are encouraged. 



Building Strong Linkages with Other Higher Education Institutions 

Western believes that strong linkages with other higher education institutions foster a spirit of collaboration and 
cooperation and provide opportunities for networking, communicating, and learning. These linkages with other two- 
year colleges as well as with four-year universities help the college stay current and at the forefront of educational 
change initiatives. Western fosters its linkages by its active involvement in a variety of consortia. 

Each of these consortia serves a different purpose and allows for a wide range of involvement by a large number of 
college administrators, faculty, and staff. One of the college’s “Vital Few” identified as part of the AQIP process is 
Valuing People. Providing opportunities for all staff members to participate in college-wide teams is one of the ways 
Western integrates its continuous quality improvement philosophy throughout the college. A unique aspect of team 
involvement at Western is the active participation of District Board members on many college teams. This participation 
provides Board members with the opportunity to work side-by-side with college staff, thereby enhancing commu- 
nication between the District Board and college employees. 

Western was one of the first colleges to be accepted into NCA’s Collaborative Quality Colloquia. This Colloquia joined 
Western with four other colleges: Eastern Iowa Community College, Lakeshore Technical College, Southern Illinois 
University, and Fort Hayes State College. Our common bond with the other consortia members is our interest in using 
the AQIP process as an alternative reaccreditation format and in collaborating with each other in the areas of 
benchmarking and data analysis. We have met one time since the NCA meeting and continue to communicate via 
a listserve and individual contacts with members of the other colleges. 

Another linkage Western has found to be beneficial is the AQIP Strategy Forum that brought together six different 
institutions in November 2000. Western was linked with Alexandria Technical College, Terra Community College, 
Eastern Iowa Community College, Capella University, and New Mexico State University for a three-day forum in which 
we learned more about the AQIP process and each other. We participated in activities designed to help us focus our 
continuous improvement efforts on three to four “Vital Few” priorities that we will measure, track, and report to NCA 
over the next three years. The collaborative spirit of the forum helped affirm the college’s quality journey and has 
resulted in follow-up meetings and communication with several partners. 

A local higher education linkage in which Western has been actively involved over the past five years is the La Crosse 
Medical Health Science Consortium (HSC]-a partnership among the three higher education institutions in La Crosse 
and the two major medical centers. The HSC is a national model of collaboration and cooperation and is meeting both 
the educational and health care needs of the Coulee Region. The HSC has fostered interest in developing more two- 
plus-two degree programs and in working together to meet both student and employer demands for health care 
workers. 

Western is also a member of the Consortium for Community College Development, a program sponsored by the 
University of Michigan. Along with nine other two-year colleges. Western is participating in a year-long Strategic 
Thinking Forum. A team of college staff is attending four two-day meetings designed to help the college learn how 
to think strategically. Collaboration with other college teams fosters a spirit of creativity and community and provides 
college staff members with yet another opportunity to bring back ideas that will enhance the college’s quality 
initiatives. 

Most recently, Western has been accepted into the Continuous Quality Improvement Network (CQIN).This organization 
is a leader in the use of continuous quality improvement principles in higher education. CQIN brings together the 
presidents of the various member colleges twice a year for collaboration and strategic planning. Additionally, CQIN 
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sponsors a summer institute for member institutions where teams of college staff members can learn how continuous 
quality improvement is used in the corporate world. 

It is obvious that building strong linkages with other higher education institutions is one of the values in which Western 
strongly believes and that Western uses to help chart its course to academic quality improvement. Western believes 
that it can learn much from its colleagues and that the exchange of ideas, energy, and camaraderie all contribute 
positively to its quality initiatives. 



Focusing Strategic Planning Efforts 

One of the most significant ways in which Western has charted its course to academic quality improvement is to focus 
and align its strategic planning efforts. The college has refined and improved its planning process over the past several 
years, and during the past year has revised its mission, vision, value, and belief statements. After completing this process, 
the college sharpened its focus on what it is really about: enrollment, retention, and student learning. In other words, 
the college asks itself three questions: Do we get them? Do we keep them? Do they learn? By simplifying priorities and 
aligning them with these three key areas, the college has also focused its data collection and analysis processes. 

Western’s goal is to collect, analyze, track, and benchmark data in the areas of enrollment, retention, and student 
learning. Western believes that by prioritizing the data it collects, it will simplify its management information processes 
and will be able to report meaningful data results that can be used for continuous improvement purposes. Western 
is committed to establishing baseline data in these three key areas, to identifying comparable data sources with whom 
it can compare our results, and to establishing benchmark and best-in-class data for which it can set stretch targets. 

Western has been tracking its enrollment data for several years and has an excellent system that follows student 
enrollment over a several-month period and compares enrollment figures with past semesters and years. These data 
have helped the college achieve its priority of attaining a stable enrollment even in times of high employment. Specific 
enrollment strategies have been implemented based on data analysis of past enrollment patterns. The next step is 
to seek comparative data with respect to enrollment patterns at similar institutions and to set benchmarks based on 
best-in-class practices. 

One of the areas of focus for the 2000-01 academic year is to increase student retention. Retention data have been 
collected for years, but have not been organized into a format that can be used for decision-making. Currently, the 
college’s EnrollmentTeam is working to develop a retention database similarto the one developed to track enrollment. 
These data will be used to track and trend retention at both the college and program levels. Specific retention 
strategies will then be developed by faculty to meet the needs of students in a variety of different programs. 
Benchmarks will be set based on retention data from consortia partners. 

Collecting and analyzing data on student learning is another area of focus for the 2000-01 academic year. Preliminary 
data were collected on student achievement of technical program outcomes during spring 2000. These data will serve 
as baseline data fortracking student learning in each major. Additionally, preliminary data were collected on student 
achievement of college-wide learning outcomes during spring 2000. These data will also serve as baseline data for 
tracking student learning in the general education area. The college is currently investigating additional ways to measure 
student learning in both general education and in the major. In particular, standardized tests are being considered so 
that the college can have comparative data to benchmark student learning results and to set stretch targets. 

Realistically, the collection and analysis of data in these three key areas will take considerable time, resources, and 
commitment. Western is determined to measure its effectiveness with respect to enrollment, retention, and student 
learning. It is only by determining baseline, comparative, and benchmark metrics that the college can continue to 
effectively chart a course to academic quality improvement. 



Using Benchmarking as an Integral Component of Academic Quality Improvement 

Strategic planning is used throughout higher education to address such issues as: 




o 

o 



Who are our potential customers? 

What changes are occurring within our marketplace, and how will we adapt? 
What are we doing to effectively meet the needs of our stakeholders? 
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Benchmarking can significantly impact the effectiveness of higher education by providing assistance in reaching 
those strategic goals. Benchmarking is the process of identifying and analyzing practices that produce desired results 
set forth by an organization. Benchmarking begs to ask the questions: 

o How good are we (who says}? 

o How good can we be (compared to whom}? 

o How do we get better (who is the best}? 

Benchmarking can take on several forms, including internal, competitive, and best-in-class. Internal benchmarking 
focuses primarily on in-house practices or processes within the college to achieve improvement. Progress can be 
consistently demonstrated over a period of time compared to a baseline. Often these improvements are process 
measures that ultimately affect outcome measures. Process measures are usually unique to specific segments of the 
college organization. Examples of internal benchmarking may be program retention, length of time to remedy a work 
order, or percent of direct high school enrollment. 

Competitive benchmarking seeks to identify comparable measures with others. These comparable measures should 
equate “apples to apples.” The measured data need to be commonly defined so the comparison is accurate. Often 
the organization compares itself to competitors, but competitive benchmarking may also occur internally by 
comparing like processes within the organization. Common examples of competitive benchmarking include length 
of admissions process, enrollment, and graduation rates. 

Best-in-class benchmarking is a continuous, ongoing effort to discover the superior or leading practices of others. 
Often these processes are revealed as a result of competitive benchmarking. Best-in-class benchmarking requires 
the expansion of horizons, being conscious not to dismiss another college for “being too different from us”-why can’t 
a two-year technical college model outstanding processes or practices on those of a four-year private college? It also 
entices colleges and universities to search outside higher education for stretch targets. Business, industry, and health 
care excel in processes similar to those used in higher education and can serve as a prototype for changing and 
improving the way a college or university does business. 

As a result of the AQIP process, Western will set targets for improvement for next year and will project stretch targets 
for subsequent years. Western could set realistic, safe targets based on past performance but has decided instead 
to set stretch targets. This decision will most likely require modification of present processes and practices to 
achieve those goals, thereby enhancing and fostering the continuous improvement of key college processes. 
Participation in consortia such as the Collaborative Quality Colloquia, CQIN, and the Strategic Leadership Forum 
provides the networking needed to identify comparable and best-in-class data. Benchmarking will augment 
Western’s Continuous Quality Improvement Initiative. Programs and sen/ice areas will establish internal benchmarking 
to monitor individual effectiveness, while competitive and best-in-class benchmarking will help establish Western as 
a leader in higher education. Western views benchmarking as an essential element of its academic quality 
improvement journey. 



Strengthening Commitment to Students, Key Stakeholders, and Staff 

When Western voluntarily elected to choose the AQIP process as an alternative format for NCA reaccreditation, it was 
vital to demonstrate to college staff the relationship between the AQIP process and its existing continuous quality 
improvement initiatives. It was especially important for college staff to understand how the seven Baldrige criteria 
align with the nine AQIP categories. The compelling question was, how does the college ask already over-worked 
employees to become involved in the process? By incorporating the AQIP process into the work of existing college 
teams and committees, Western has been able to garner substantial college-wide support for this new approach to 
reaccreditation. In particular, the prior commitment to student success and to measuring student learning aligned 
with the AQIP category-Helping Students Learn. There was a need to incorporate the process into the work of existing 
college committees. 

Participation in the AQIP process required Western to select its “Vital Few”-those areas selected to track and measure 
over the next three years. Western chose four of the nine AQIP criteria as the major foci of its continuous quality 
improvement efforts for the next three years. Selecting the “Vital Few” was a collaborative effort by the Leadership 
and Strategic Planning committees at Western. It was imperative to align the “Vital Few” with the existing Strategic 
Priorities as shown in the following chart. 



O 
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AQIP “Vital Few” 


Western’s Strategic Priorities 


Helping Students Learn 

Understanding Students’ and OtherStake- 
holders’ Needs 


Measuring Effectiveness 


Maintain a stable enrollment by offering a comprehen- 
sive, cost-effective mix of programs and services that 
reflect the dynamic educational needs of the District 


Helping Students Learn 


Enhance Student Success 


Helping Students Learn 


Measure and Improve Student Learning 


Understanding Students’ and OtherStake- 
holders’ Needs 


Increase Student Satisfaction 


Valuing People 


Increase Employee Satisfaction 



The AQIP criteria Helping Students Learn and Understanding Students’ and OtherStakeholders’ Needs relate to more 
than one strategic priority; Valuing People aligns with a single priority; and Measuring Effectiveness supports all the 
strategic priorities. 

Choosing to link the AQIP “Vital Few” to the college’s strategic priorities reinforced Western’s beliefs and fostered 
buy-in and acceptance by faculty and staff. At Western it is believed that excellence is related to enrollment, retention, 
and student learning, or the mantra: “Do we get them? Do we keep them? Do they learn?” The readiness to share 
Western’s “Vital Few” publicly demonstrates its long-term commitment to the process that in turn supports the 
institutional goals. Faculty and staff are less inclined to view NCA reaccreditation as additional work, but instead to 
view it as real work” that validates the goals by striving to create an excellent product and environment for all 
students, employees, and community stakeholders. 



Enhancing a Key College Value: Working Together In a Healthy Environment Where 
Creativity, Humor, and Fun Are Encouraged 

Few conditions have a greater impact on an organization than internal conflict. Yet many leaders avoid focusing on 
conflict within their organizations, perhaps because it is one of the most difficult of all areas to address. Ironically, 
many leaders realize that the ability to effectively address conflict will enable the organization to respond to emerging 
challenges more than any other factor. One of Western’s key values-working together in a healthy environment where 
creativity, humor, and fun are encouraged-plays a significant role in the college’s ability to successfully chart its 
course to academic quality improvement. 

It is true that human beings disagree; this is a normal condition in the workplace. It is especially true that human beings 
in a complex organization will disagree over issues that arise as part of a normal work environment. When that work 
environment also includes a collective bargaining unit, an additional element is added that can contribute complexity 
and confrontation. However, it is also a normal human condition to seek conflict resolution when disagreements arise. 
Successful organizations understand howto balance these seeming polarities. 

One of the ways Western has been able to meet some of these challenges as they relate to creating a healthy work 
environment has been for organizational leaders to communicate this balanced vision as an expectation for the 
organization. Many leaders shy away from addressing the second element of the equation-resolving and managing 
conflict. By doing so, they inadvertently send the message to employees that any level of conflict is normal and 
expected. Western has consciously attempted to accept the fact that internal conflict is inevitable; yet, because one 
of Western’s “Vital Few” is Valuing People, it is critical to also convey the message that conflict is manageable. 

Some of the simple steps Western has taken to address the issue of internal conflict enforce one of the critical 
elements that define the Western Way: caring. Other organizations can follow some of these simple suggestions that 
help address internal conflict: 

1 . Leaders should clearly spell out in the organizational value statements that everyone shares the responsibility 
for conflict resolution. While it is not possible to completely eliminate conflict, organizations will expend far 
less energy in addressing problems if they are not passed along to someone else to solve. 
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2. Supervisors should receive training in conflict resolution techniques. Frequently, early intervention can clear 
up misunderstandings. Supervisors are most often in the position of recognizing conflict at the early stages 
and can effectively intervene. 

3. Leaders should evaluate the system for rewards and recognition. Programs such as the “employee of the year” 
(though well-intended] can often produce negative side effects that outweigh the positives. Ironically, the 
individual performance that is recognized is often, in reality, the result of a team effort. For this reason, 
individual recognition often leads to resentment. In fact many organizations have stopped using pay-for- 
performance incentives and even traditional performance appraisals for similar reasons. Internal competition, 
as a rule, is divisive. It is one of the reasons Western has abandoned all of its “employee of the year” awards. 
To be certain, each organization is unique. Consequently, there is no one-size-fits-all approach to addressing 
rewards and recognition. Nonetheless, this is a key area in the effort to reduce internal conflict. 

4. Leaders must walk the talk. When leaders find themselves drawn into conflict and appear to show bias within 
the organization, they can likely anticipate others to do much the same. That’s just the way it works. Actions 
speak more loudly than words. 

5. Finally, maintain a sense of humor and have fun. Encourage others within the organization to have fun as well. 
This can be the best medicine to reduce conflict and focus on a resolution. 

Most leaders recognize the need to build flexibility into their organization. By building an expectation of cooperation, 
teamwork, conflict resolution, and fun into Western’s value system, the college has also built in organizational flexibility. 
Flexibility leads to success and to an organization’s ability to respond to a variety of situations with agility. Most importantly, 
Western’s approach to conflict management is based on one of its primary values-caring for its employees. 



Conclusion 

Western has charted a course to academic quality improvement by integrating its continuous quality improvement 
initiatives and its preparation for North Central Association (NCA] reaccreditation. By aligning its ongoing continuous 
improvement activities with the AQIP process, Western has been able to simplify its processes and to focus its human 
and other resources more effectively. Central to the college’s ability to respond to its customer needs while 
maintaining a constant, consistent focus on quality is its dedication to critical elements that define the Western Way: 
collaboration, communication, caring, and commitment. 
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Show Up. . .Play Nice. . .Work Together: 
Preparing Students for the 
High Technology Workplace 



Ray Turner 
Toni Glasscoe 



The STAR Institute in Lansing, Michigan, was launched in fall 1998 to address several needs: 

° Business and industry in the Lansing Community College [LCC) service area needed a steady supply of 
workers with high technology skills. 

o Area high schools were largely ill equipped to prepare students in several key high technology career paths 
and were looking for ways to provide their students with this option. 

o Students, while ready to learn high technology skills, were woefully unprepared for actual participation in a 
contemporary workplace. 

The STAR Institute was created by Lansing Community College to meet these needs. The college’s partners include 
the Lansing Public Schools and a collection of local and regional businesses. Enrollment is open to any student in 
the college’s service area, with the cost of enrollment paid by the student’s home school district. Not all schools in 
the service area are willing to support their students’ participation in STAR. 

Students attend STAR Institute for half of the day and their home schools for the other half. 

STAR Institute curricula mirror curricula on the main LCC campus, and students earn LCC credit for their STAR work. 
They have an LCC transcript that reflects coursework identical to the transcripts of traditional college-age students 
taking similar programs on the main campus. STAR classes vary from main campus classes in that students spend 
more time in each class, work in a cohort group for one to two years, and learn workplace skills along with the technical 
training of the curricula. 



The Problem 

Integrating the necessary technical training and the workplace skills training in ways that are effective and meaningful 
to high school students presents unique problems: 

o High school students are high school students. Carrying six to nine college credits and three classes at their 
home school, and learning how to function in the workplace and make it to the prom on time is asking a lot 
from a 16-year-old. To accomplish our goals would require new approaches. 

° Students and teachers alike are accustomed to the basic patterns of traditional classrooms, patterns that are 
not flexible, include burdensome redundancy that limits creative use of time, and have little built-in problem- 
solving potential. 

o The traditional course structure and class schedule do not promote meaningful, integrated learning. They are 
designed to separate learning into digestible chunks. The STAR Institute needed to pull those chunks together, 
gain efficiency in using classroom time, and increase flexibility dramatically. 
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What We Tried 

During the STAR’S first year Cl 998-1 999), it became clear that students were not going to learn workplace skills by 
adding a few “howto write your resume” seminars and taking a few field trips. Initial attempts at using this approach 
met with dismal failure and significant student rebellion. We discovered that the students’ home schools had already 
shut down the avenue of seminars and workshops by exposing their students to a series of very bad seminars and 
workshops that were not related either to their fields of study or their career plans. This realization led faculty, in 
consultation with students, to Project Based Learning. 



Project Based Learning (PBL) 

PBL is not a new concept and it has a very good track record in many settings. Connecting PBL with the students’ 
technical training, setting aside specific time for PBL work, and trying to create as much as possible real-world 
projects, PBL began to give students a sense of how to apply their technical skills to real-world problems and how 
to work together to plan and solve problems. 

During the second year of PBL at STAR [1 999-2000), it became clearthatthe workplace skills most lacking in students 
were communication skills. STAR Institute advisory councils reinforced this conclusion vigorously. Forthe current year 
C2000-2001), all STAR curricula were revised to include coursework in oral communication in the workplace and 
technical writing. Adjustments to technical course offerings were made, but the credit load was not diminished. 



More Work — Still Only Twenty-Four Hours 

STAR faculty and staff looked ahead to the expanded course requirements of 2000-2001, with genuine concerns 
about student workloads. Curricula required students to do more work in the same time. How could we ask them to 
do technical coursework, PBL, and now two additional communication courses and still make the program work for 
them? 

We concluded that we needed to aggressively pursue three strategies: comprehensive integration of communication 
courses, technical courses, and PBL; a vigorous application of the use of the wide range of quality learning tools 
typified by those promoted by David Langford; and a thorough student orientation at the beginning of the year. 



Integration of Communication and Technical Courses 

Still in process, this strategy has had mixed results. The theory, of course, is to combine assignments or activities in 
two or more classes to accomplish multiple learning outcomes from the same effort and time. The close coordination 
between faculty needed to do this well is difficult to achieve. Finding the time, overcoming historic patterns, and 
scheduling learning units to coincide are all problems that STAR faculty and staff are struggling with to improve and 
increase the integration of coursework. 



Quality Learning Tools 

In June 2000, at the end of STAR’S academic year, most faculty and staff attended a week-long seminar with David 
Langford. Langford’s approach to using traditional quality tools in the classroom is proving to be a good fit for STAR 
faculty and students as they plan together and solve problems. The tools have provided a strong base for faculty to 
think and work together on integration problems. During the summer of 2000, many faculty used these tools to prepare 
for the third element of our improvement plan-a thorough student orientation at the beginning of fall semester. The 
STAR Institute presentation at this NCA Annual Meeting will focus on the use of these tools in the context of our work 
on technology and workplace skills. 



STARGate — Getting Students Ready forthe STAR Experience 

STAR faculty and staff set aside two full weeks at the beginning of fall semester to orient students to the skills they 
would need and the expectations that come with STAR participation. This two-week session was highly structured 
and involved all STAR faculty and staff, most of whom were present for the entire orientation. Various faculty and staff 
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led different portions of STARGate and generally had the assistance of several others for the application activities. 
Students learned about STAR systems, about STAR technology resources and how to use them, about Lansing 
Community College and its services, and about quality learning tools. 

If STARGate were a research project, it would be one that came with a built-in control group. The relevant behavior 
and achievement of second-year students Cwho did not have this orientation in their first year) can be compared to 
that of first-year students Cwho have never participated in STAR without the orientation.) In preparation for next year, 
faculty and staff will be assessing STARGate as it plans for the next round of improvements. 



Conclusion 

Taken together, curriculum integration, broad application of quality learning tools and thorough orientation of 
students hold the promise of delivering higher quality learning. Addressing both a wide range of subjects and their 
effective application to the workplace must be accomplished for STAR students to be successful in our demanding 
program. The STAR faculty and staff, as they have for two-plus years now, continue to measure and evaluate the 
effectiveness of this experiment and refine it to improve their service to students. 



Ray Turner is Director of the STAR Institute at Lansing Community College in Lansing, Michigan. 

Toni Glasscoe is Coordinator of the STAR Institute at Lansing Community College in Lansing, Michigan. 
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Choosing AQIP: 
To Be or Not To Be 
(a Participant)? 



Terry Kuhn 
Paul Gaston 
Karen Budd 

Kent State University, a doctoral/research-extensive institution, consists of a large, diverse residential campus and 
seven regional campuses. The Kent campus provides baccalaureate, master s, and doctoral study opportunities, while 
the Regional Campuses serve specific community needs with associate degree programs in more than twenty-four 
technical and business fields. The eight-campus system covers a geographic area the size of Connecticut. Kent State 
offers a full range of academic programs with supportive strengths in focused research and service areas. 

The total enrollment of Kent State University reflects 6000 residential students and 26,000 commuter students who 
approach its campuses from nearby apartments and from cities including Akron, Cleveland, and Canton. More than 
30,000 Kent State students are Ohio residents, and the remaining 2000 come from 46 states and 33 countries. There 
are 2600 adult students and about 1700 students transferred to Kent from other institutions of higher education. 
Sixty-one percent of our students are female, and 87 percent are Caucasian. Kent State’s faculty is 63 percent male 
and 88 percent Caucasian. Kent State undergraduate students graduate with more indebtedness than the national 
average, and they work more hours per week than the Ohio norm. 

This description shows Kent State to be large and complex, with characteristics of traditional residential and 
innovative metropolitan institutions. Why, given this degree of complexity in its institutional mission, did Kent State 
decide to apply to be an AQIP Partner Institution? 



NCA 2000 Annual Meeting 

A small contingent from Kent State attended the NCA April 1 -3, 2000, Annual Meeting to learn about the Academic 
Quality Improvement Project [AQIP). With our next comprehensive evaluation in 2004, the timing seemed optimal to 
investigate this alternative accreditation process. We attended sessions such as “Articulation of the Baldrige Criteria 
to Meet the Self-Study Guidelinesfor NCA,” “The Journey to Quality,” “Introducing the Academic Quality Improvement 
Project's AQIP Alternative,” “Incorporating the Malcolm Baldrige National Quality Criteria into Your NCA Accredita- 
tion Process,” “The Journey to Quality: Is It Far From Here?” and “What We re Learning About Quality and 
Accreditation.” These sessions developed awareness and understanding of NCA’s purpose to help its institutions 
improve by initiating the Academic Quality Improvement Project. 



Partnership Request 

As a result of attending the Annual Meeting, the following recommendation was forwarded to Provost Paul Gaston 
on April 4, 2000: Kent State University should investigate the Academic Quality Improvement Project [AQIP) as an 
alternative accreditation method within the North Central Association of Colleges and Schools Commission on 
Institutions of Higher Education. He reacted with wry humor, saying, “I thoughtyou’d never ask.” President Cartwright, 
too, whose leadership term for AAHE coincided with rising interest - and considerable skepticism - in the higher 
education community about outcomes assessment and continuous improvement practices, was enthusiastically 
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supportive. Utilizing an informal advisory group of faculty experts from throughout the university, a Partnership 
Request was completed through the spring and early summer and sent to NCA. Kent was accepted as an AQIP Partner 
Institution in September 2000. 

When administrators and faculty reviewed the AQIP criteria, much skepticism was expressed. Cautionary anecdotal 
accounts of similar efforts in program accreditations were mentioned in recognition of the revolutionary potential of 
the movement and of the facts that it meant going far beyond business as usual. In fact, much of the value of the 
application process was to help us learn more about our own current efforts and the depth of expertise and experience 
that already existed on our campus in the area of continuous improvement. Most of this expertise was going into the 
classroom and into consulting with business and industry on these very concepts! 



Campus Readiness 

Assessment of the NCA General Institutional Requirements and Criteria contained in Kent State’s 1994 Team Report 
suggested that our accreditation status appeared secure and that no serious shortcomings jeopardized our regional 
accreditation. Following this determination, public presentations to build awareness about AQIP were made to the 
President’s Cabinet, Chairs and Directors Council, Academic Administrative Council, Provost’s Faculty Advisory 
Council, University Teaching Council, and the College of Nursing faculty. There was good support in each of these 
groups for pursuing the AQIP alternative accreditation procedure. Some questions from faculty and administrative 
staff did evidence an appropriate degree of supportive skepticism. 



Questions Considered 

Several faculty asked why regional accreditation was important to the university. Answers included assurance of 
quality education for students, accountability to stakeholders, public credibility, the contingency of regional 
accreditation to obtain federal funds and student loan assurance, and transferability of student academic credit. 

Perhaps the most pressing question from these groups was, “What is this alternative accreditation process?" It was 
suggested that AQIP would emphasize continuous improvement, the seven Baldrige criteria, a focus on students and 
their learning, and a systematic feedback program. Rather than a several-hundred-page self-study every ten years, 
Kent State would be expected to write an annual fifty-page report that would examine both processes and outcomes! 

As questions arose about why we should participate in AQIP, we tended to rely on the materials provided by NCA. 
For example, AQIP provides a framework for assessing, measuring, and improving performance in our learning, 
operations, and financial procedures. AQIP also provides a mechanism to get feedback from peer institutions that will 
help identify strengths and opportunities for improvement. Furthermore, we felt that we already met the GIRs and that 
there would be much more to be gained from this alternative process. We also hoped that, as an early participant, 
we would have an opportunity to help shape the AQIP process. Lastly, we felt there might be an opportunity to better 
focus a distinctive identity for our institution. 



Assessing Readiness for AQIP 

Completing the AQIP Partnership Request caused us to evaluate our readiness for participation. There was agreement 
that while we had several experiences with quality programs in three divisions, no comprehensive, institution-wide 
TQM or similar movement had been attempted. We had just completed rewriting our strategic plan with the personal 
involvement of President Cartwright. We had received several outcomes-based Selective Excellence awards from the 
Ohio Board of Regents. We have a systematic, periodic review process for assessing graduate programs. Many of our 
academic programs participate in accreditation procedures that emphasize similar principles (business, nursing, 
education). Kent had also received the Governor’s Excellence in Workers’ Compensation Award. In addition to 
receiving external recognition, Kent State also gives formal internal recognition to distinguished teachers, to 
outstanding researchers, and to alumni, staff, and individuals who make outstanding efforts on behalf of students. 

Our executive officers are well-versed in continuous quality efforts. President Carol Cartwright has provided 
significant leadership to quality improvement initiatives at the national and state levels as well as on behalf of Kent 
State University. Provost Gaston has provided workshops and participated in national meetings on quality 
management. Vice President Creamer began his career as an operational and compliance auditor with the U.S. 
Department of Health, Education and Welfare using tools and techniques similar to Deming’s Total Quality 
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Management. Kent State has used the Student Satisfaction Survey, the Cooperative Institutional Research Program 
new freshman survey, NASPA benchmarking of student services programs, the ACT Survey of Academic Advising, 
the National Survey of Student Engagement, and an externally conducted Cultural Self-Study. We have consistently 
sought data about and from our students, formed committees to review study results, and developed strategies to 
address problems identified. 



Current AQIP Efforts 

As this paper is being written in late December 2000, plans are being prepared to pursue a Level II application for 
the Ohio Award for Excellence (OAE). Kent State’s AQIP Liaison and a faculty member have been trained and 
appointed as examiners for the Ohio Award for Excellence. The OAE uses the same criteria as AQIP and the national 
Baldrige award program. 

Plans are being formulated to provide resources to support the AQIP effort and to train administrators and faculty in 
the concepts and procedures to enable broad participation. Special care is being taken to develop a macro view for 
our institutional involvement while keeping that macro view responsive to academic programs. Institutional 
expectations for course and program objectives are being made explicit and embedded in the regular curricular 
processes. We are considering hiring outside consultants to help us conceptualize the establishment of systems, 
processes, and structures to stimulate and audit institutional quality improvement efforts. We feel it is particularly 
important to engage the academic sector in continuous improvement because most of our past efforts have been in 
non-academic sectors of the university. 

An informal AQIP advisory committee will continue, even while a more formal AQIP Implementation Steering 
Committee is created. The challenge will be to ensure that reporting and assessment activities take place at an 
institutional level with meaningful involvement from division, college, and program levels. 



Conclusion 

While we have approached our decision-making process for joining the Academic Quality Improvement Project 
through the template provided by NCA, this description does not convey the suspense and excitement of the process. 
We moved from interest pursued at the 2000 NCA Annual Meeting, to enthusiasm generated by those who attended, 
to doubt (almost dismay) when the many issues were considered, to hope in the realization that we had experience 
and expertise to engage meaningfully, to celebration when we found out we were accepted as an AQIP Partner 
Institution, to realism over the fact that we now have it to do. The decision made on this roller-coaster ride seemed 
clear from the outset. That is because there is perceived potential for making significant and positive changes in Kent 
State University’s institutional culture through the process methodology embodied in AQIP’s continuous improve- 
ment project. 



Terry L Kuhn is Vice Provost and Dean of Undergraduate Studies at Kent State University in Kent, Ohio. 
Paul Gaston is Provost at Kent State University in Kent, Ohio. 

Karen Budd is Associate Professor, College of Nursing, at Kent State University in Kent, Ohio. 
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AQIP as a Change Agent 
in Higher Education: 

Lessons Learned by a Charter Institution 



Less than two years ago, Iowa Valley Community College District [Iowa Valley) embarked on a reaccreditation/ 
organizational change journey with three goals in mind: (1) to obtain Iowa Valley’s first district accreditation; 
(2) to achieve that accreditation using the Academic Quality Improvement Project [AQIP) paradigm; and (3) using the 
AQIP tool to identify, implement, and sustain organizational change. Goals one and two were accomplished last 
October when Iowa Valley was selected as one of fourteen charter members of AQIP and its accreditation was 
extended to 2007-2008, with a possible additional extension to 2015. Goal three is an ongoing process, and we are 
near the end of the second year. 

District accreditation was a daunting task for a couple of reasons. First, for more than thirty years, Iowa Valley’s two 
distinctive, semi-autonomous, credit-granting units (Ellsworth Community College in Iowa Falls and Marshalltown 
Community College in Marshalltown) had been accredited individually. Second, we decided to pursue accreditation 
using the AQIP paradigm while it was still being developed by NCA. Like the crew of the Starship Enterprise, we were 
going “where no [educational institution] has gone before.” 

Since 1996, Iowa Valley has been using district Total Quality Management (TQM) teams to address salient 
institutional issues. One of these teams, tasked with developing the process for Iowa Valley's district accreditation, 
recognized the viability of using the Malcolm Baldrige Criteria for Excellence in Education for our self-assessment 
because it would further our continuous quality improvement efforts. NCA’s AQIP paradigm, based on the philosophy 
of the Baldrige criteria, allows institutions to link ongoing quality improvement efforts to NCA accreditation. Because 
of the similar quality emphasis, we felt we could easily transition from Baldrige to AQIP once it was completely 
developed. 

Eighteen months later, with our AQIP partnership accepted and our reaccreditation extended seven years, the NCA 
AQIP teams were completing and documenting our Baldrige-based self-assessment. As the Steering Committee, the 
team leading the reaccreditation effort, reviewed the team reports, it realized it had failed to evaluate its process and 
took steps to remedy the oversight. 

At the same time the Steering Committing was evaluating its process, the director of that committee attended a 
leadership workshop where Kotter’s article, “Eight Steps to Transforming Your Organization” G Harvard Business 
Review, March-April 1995) was discussed. Kotter indicated that as institutions move through the eight steps of 
transformation, errors could occur. When we looked at our transformation efforts, we realized errors did occur. 
Kotter’s eight errors are: 1 

1 . Not establishing a great enough sense of urgency 

2. Not creating a powerful enough guiding coalition 

3. Lacking a vision 

4. Under-communicating the vision by a factor of ten 

5. Not removing obstacles to the new vision 



Laura D. Browne 
Cindy Green 
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6. Not systematically planning for and creating short-term wins 

7. Declaring victory too soon 

8. Not anchoring changes in the corporation’s culture. 

The following is a discussion of the lessons we learned from the errors we made. 

The first error we made was ‘’not establishing a great enough sense of urgency” [p . 60). The mandated district 
accreditation referenced earlier was the basis of our sense of urgency. Limiting this urgency to accreditation only was 
a potentially fatal error. The consequences of failing to be reaccredited were familiar to most of Iowa Valley’s 
employees; therefore, establishing a sense of urgency regarding reaccreditation was relatively easy. We were able 
to establish the need to complete reaccreditation Using a quality model, since it complemented our ongoing quality 
efforts. The error we made was not establishing a sense of urgency for carrying on continuous organization change. 
We assumed that past TQM efforts had established awareness that individual team projects impacted the overall 
institutional change process. This was a flawed assumption. Iowa Valley TQM team members did not recognize that 
their projects were pieces of a bigger puzzle: organizational change. Because the Steering Committee erroneously 
assumed this connection had been made, little attention was given to establishing a great enough sense of urgency 
for maintaining the momentum for continued organizational transformation. 

Our second error was “under-communicating the vision” [p. 63). Our purposeful use of paper and electronic media 
to communicate the vision was not as effective as we had hoped. We eventually realized that many people were not 
reading our communications. In some instances, the technology of the Intranet was a deterrent; in other cases, people 
were overwhelmed by the amount of information provided. We found people preferred to participate in meetings or 
have informal [purposeful) conversations about AQIP accreditation rather than read Intranet postings, minutes of 
meetings, quality manuals, or newsletters. 

Our last error was “declaring victory too soon” [p. 66). Kotter warned that premature celebrations would kill 
momentum, and when Iowa Valley’s president announced that we had achieved accreditation, we lost much of the 
momentum necessary to sustain our organizational change efforts. Many District employees, grounded in the 
traditional accreditation model, believed their work was done. So much for continuous organizational change! 

We have learned a great deal; most of which can be summed up in this quote from Dr. Stephen Spangehl, NCA-AQIP 
Director: 



Most of the institutions I have talked to that have been doing systematic quality improvements tell me that they 
didn’t invest as much as they should have ... at the front end in cultural shifting, in training people to think in 
new ways, to think in terms of process, to think about working in teams of a cross-institutional nature where 
people from many different kinds of positions brought their perspectives to bear on a common problem [Kent 
Sate University, 9-8-00). 

Kotter noted that organizational transformation often takes five to ten years [p. 67). While the evaluation of our 
Steering Committee’s leadership process showed our mistakes, it also showed many of the things we did well. Thus, 
Iowa Valley’s transformation journey continues. 



Laura D. Browne is Associate Dean of Developmental Services at Iowa Valley Community College District in 
Marshalltown. 

Cindy Green is Dean of Instruction at Iowa Valley Community College District in Marshalltown. 
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SWIMM, Don't Flounder: 
Two-Year Institutions Respond to 
NCA's Total Quality Initiative 



Sherrill L. McCormack 
Linda Johnson 

What Is SWIMM? 

In 2000, five two-year institutions accepted NCA’s invitation to explore the AQIP model for accreditation and joined 
a colloquium, SWIMM-an acronym formed by the first letters of the five states represented in the colloquium. The 
colloquium encourages exchange of total quality models and offers a forum for discovery about benchmarking and 
data sharing. 

SWIMM is comprised of Southeast Technical Institute in South Dakota, Chippewa Valley Technical College in 
Wisconsin, Iowa Valley Community College District in Iowa, Alexandria Technical College in Minnesota, and Crowder 
College in Missouri. Each institution is at a different stage of quality improvement, and each has used a different 
technique to achieve its current status. Three of the members are technical institutions, while two are a combination 
of liberal arts and technical studies. 



Colloquium History 

Initially, the Self-Study Coordinators and members of the college assessment teams from the institutions exchanged 
email addresses, and early correspondence began. Then, Alexandria Technical coordinated a conference call forthe 
institutions to learn about one another. This allowed each representative to explain his or her institution’s quality 
process. The email correspondence continued, with Iowa Valley agreeing to coordinate a meeting forthe group during 
the NCA Annual Meeting in April 2000. The group increased to its current membership of five institutions by the time 
of the Annual Meeting in 2000. 

This first face-to-face meeting was also useful in that the coordinators and other administrators, including presidents, 
from the five institutions met to discuss direction, purpose of the group, and assisting NCA-AQIP in the development 
process. Institutions discussed the type of quality assessment tool being used by each. These tools ranged from 
Malcolm Baldrige to Pacesetter to state quality assessment tools. 

To further these discussions, a follow-up meeting was planned at a central location for June 2000. Iowa Valley hosted 
the one-day meeting at their Marshalltown facility. During this meeting, details of each institution’s process were 
presented. Results of meetings with NCA-AQIP were reported, and a conference call with Eleanor Harrison provided 
answers to further questions. Additionally, Iowa Valley volunteered to host a listserve for participants, and Crowder 
College volunteered to host a web site, SWIMM.org, to assist with data comparisons. This site allows the institutions 
to exchange results relating to the nine AQIP Criteria. 

The 2001 NCA Annual Meeting will be the site forthe third face-to-face meeting. Technology has proven essential 
for communicating, but these in-person meetings are also necessary. 
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Challenges of Colloquia Participation 

Embracing total quality principles represents a major paradigm shift from the traditional organizational structures of 
most academic institutions. Therefore, participants in the colloquium had to carefully consider whether the new 
accreditation model was a viable option for their institutions. Members shared details of their quality processes- 
successes and failures. Although sharing failures can be difficult, the struggles of each organization have been helpful 
to other colloquium members. Trust is also an important and essential aspect of membership. 

Participants discussed several issues/questions: (1) What traditional organizational structures can be incorporated 
into this new model? What structures are in place? (2) What background and training did the faculty and staff in your 
organization have? What will they need? (JO What quality self-assessment tool is the most effective? Does the type 
of tool matter? Can we use state quality awards for the NCA process? 

Benchmarking is one of the primary goals of the group, and it remains one of the greatest challenges. Few two-year 
institutions have won national quality awards, and few comparative data are available. Some colloquium members 
are participating in state quality awards and using those self-assessments as tools for accreditation. 

Additionally, as the group moves forward, we are considering: (1) What information is feasible to share? (2) How do 
we share politically sensitive data? (3) How public should we make this information? 

Initially, several indirect measures such as employer, graduate, and student satisfaction surveys are being compared. 
As the process progresses, members will tackle the challenge of sharing direct measures of student academic 
achievement. 

Finally, due to geographical distance, institutions face the challenge of maintaining momentum. While in-person 
meetings create excitement and interest, participants return to their institutions and daily assignments. Communi- 
cation begun through the initial meetings must be continued through regular and committed use of the listserve and 
updating and checking the web site. 



Benefits of Colloquium Participation 

The most immediate benefit of the colloquium has been the exchange of experiences among the participants from 
the various institutions. Learning that other schools have followed similar paths is both informative and reassuring. 
The impetus for new ideas frequently occurs when groups can exchange openly and in a climate of trust. As the 
SWIMM members discuss their processes, we discover that some institutions are utilizing surveys some of us have 
not considered, and the willingness to share surveys is a great benefit. Since all colleges in the partnership are in 
different states, and are separated by considerable distance, there has been no sense of “competitive threat” in 
sharing performance data and results. This perspective of external partners looking at an organization is a valuable 
tool for quality improvement. 



Future Plans of SWIMM 

SWIMM members see continued benefits from participation in the colloquium. As further use of the web site occurs, 
additional comparison and benchmarking will take place. Continued communication and comparison of processes 
will help each of the institutions in their quality improvement processes. The SWIMM Team listserve and the 
SWIMM.org web site are key vehicles for accomplishing these goals. 



Sherrill McCormack is a Business Instructor, Self-Study Co-Chair, and Co-Chair of the Institutional Steering Committee 
at Crowder College in Neosho, Missouri. 

Linda Johnson is an English Instructor, Assessment Coordinator, and Self-Study Co-Chair at Southeast Technical 
Institute in Sioux Falls, South Dakota. 




164 

\ * 



162 /A Collection of Papers on Self-Study and Institutional Improvement, 2001 



Innovative Designs: 
Building Strength on Strength 



Dave Weber 
Don Supella 

Created in 1 91 5, Rochester Community and Technical College (RCTC) has the distinction of being Minnesota’s oldest 
community college. RCTC also has the distinction of being located in America’s fiftieth fastest-growing metropolitan 
area. The combination of rich tradition and high-speed economic growth offer RCTC significant challenges. 

A consolidated college, RCTC combines the best in liberal arts, technical, and lifelong learning. The college offers 70 
technical and transfer programs and serves nearly 6500 students in credit-based programs and 1 1 ,000 in workforce 
education offerings. Located at the University Center Rochester, RCTC, a scenic 400-acre campus, is nestled in the 
Mayo Run environmental corridor in southeastern Minnesota. Minnesota’s two public higher education systems 
(Minnesota State Colleges and Universities and the University of Minnesota) collaborated to create the unique 
campus called University Center Rochester. This shared campus, home of Rochester Community and Technical 
College, Winona State University-Rochester Center, and the University of Minnesota Rochester, offers learning 
opportunities that range from community lifelong learning to customized training and doctoral programs-all in one 
physical location. Rochester Community and Technical College is the primary provider of lower-division and 
customized training courses for the University Center. 

The challenge facing all higher education institutions, but especially one located in an economic and population 
“boom town,” is one of matching performance to purpose. Last year RCTC launched a strategic academic planning 
process, Innovative Designs, intended to connect master academic and strategic planning to accreditation and 
quality. At last year’s NCA Annual Meeting, RCTC shared its alternative approach to reaccreditation, an approach 
centered around the Minnesota Council for Quality’s Baldrige assessment process. Using Innovative Designs as a 
foundation, RCTC’s quality journey continues this year as we prepare for an April 2001 on-site reaccreditation visit. 
This year’s journey, Innovative Designs: Building Strength on Strength, provides evidence of Rochester Community 
and Technical College’s continued focus on using results to improve teaching and learning. 

Key to continuous improvement is an understanding of the environment or a scan ofthe landscape. Our environment 
is being shaped by changing stakeholder perceptions of value including the: 

o Demand for relevancy 

o Desire for accelerated degree completion 

o Expectation of accessibility 

o Realization that the goals of many students are not to earn a certificate, diploma, or degree 
o Request for flexibility and convenience 
o Need to balance work, family, and education 

A close look at the institutional changes that must occur to meet the challenges of a new model for higher education 
while fulfilling our NCA self-study obligation led RCTC to create our Innovative Designs approach. 



Innovative Designs 3-2-1 

“Plan, Do, Check, Act.” It is the “doing” and “acting" that can hold higher education hostage. To help assure action 
our planning and studying, RCTC adopted a 3-2-1 slogan to our Innovative Designs effort at linking planning, 
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quality, and accreditation. This Innovative Designs 3-2-1 initiative creates a quality-based approach to accountability 
that aligns academic and strategic planning with accreditation in an environment of continuous improvement. The 
Innovative Designs approach incorporates three areas of focus, two initiatives, and one goal. The triangles identify 
Innovative Designs’ three areas of focus: accreditation, strategic 
planning, and quality. The two initiatives are [1] to gain NCA reaccredi- 
tation using a Baldrige-based assessment process, and [2) to align 
RCTC with the needs of students, stakeholders, and the community 
using a strategic planning process involving both internal and 
community stakeholders. Our one goal is to continuously improve. 

The Innovative Designs 3-2-1 approach allows for the: 

o development of an integrated model aligning accredi- 
tation with strategic, academic, facilities, technologi- 
cal, and operational planning using Baldrige and the 
Minnesota Quality Award Performance Excellence 
Framework; 

o promotion of greater synergies and alignment between quality improvement, assessment, institutional 
research, and strategic planning; 

o emphasis on teaching and learning as our key system while redefining processes to support teaching and 
learning; 

o creation of a student-centered college focused on continuous improvement and shared learning; 

o development of a systematic approach that aligns short-term work plans with a long-term view of the future. 




What is the value of implementing this innovative design approach? RCTC’s unique approach promotes continuous 
versus episodic organizational assessment and review, provides a comprehensive framework assuring year-to-year 
improvement, supports continuous improvement by building on strengths, and assures that college resources are 
aligned. 

Organizing for Quality 

At last year’s NCA Annual Meeting, RCTC shared its 
unique committee structure. The structure, intended 
to support the Innovative Designs model, integrates 
traditional committees with new committees 
reflecting the Baldrige Performance Excellence 
Criteria. This out-of-this-world committee struc- 
ture links all committees and work units in a 
relational versus a hierarchical organizational 
structure. Included in the new structure is an 
Innovative Designs Committee, composed of repre- 
sentatives from other all-college committees including 
Teaching and Learning, Student and Stakeholder Rela- 
tions, Employee Relations, and Technology. Additionally, to 
facilitate the college’s desire for broad-based college-wide sup- 
port for Innovative Designs, comprehensive staff development activities and a diverse communications plan have 
been implemented. Implementation methods, successes, and opportunities for improvement resulting from our flight 
into outer space are shared in the 2001 presentation. 




FY00/FY01 

Committee 

Structure 



Innovative Designs Strategic Planning 

Shortly before our 2000 NCA presentation, RCTC formed a Futures Task Force to focus on the strategic planning 
portion of our 3-2-1 focus. The result of our strategic plan will be a more agile, flexible organization responsive to 
student, stakeholder, and community needs. A major part of our 2001 NCA presentation will be the sharing of our 
journey toward the completion of this 18-month process. The planning process included both internal and external 
stakeholders and has six primary stages. 
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Innovative Designs Planning Stage 1: 
Landscape Analysis 

Landscape analysis or an environmental scan, the 
first stage of the Innovative Designs process, was 
used to identify the market requirements of stu- 
dents and other stakeholders. Key trends were 
identified to help determine our design criteria. 
These trends were prioritized based on the influ- 
ence and impact each is expected to have on 
RCTC’s future. 



Innovative Designs Planning Stage 2: 
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Landscape 



The second stage of the Innovative Designs process was signature review. The college’s signature includes the 
statements, symbols, and the words defining the college in the marketplace. This process included a review, 
reaffirmation, and/or redrafting of the college vision, mission and belief statements, slogans, logos, and other visual 
images. As partofthis process, RCTC defined key terms and identified essential characteristics within these signature 
statements. These definitions and identifications helped establish a shared language and culture, which will support 
the identification of key performance indicators later in the planning process. 



□ “The RCTC Signature” 



Vision. Rochester Community and Technical College will be a world-class provider of technical, liberal arts, and 
lifelong learning. 

Mission. Rochester Community and Technical College provides accessible, affordable quality educational 
opportunities to a diverse community. 

Values. We believe: 

o Learning is a lifelong process reflected in an academic continuum of developmental, general, 
technical, and transfer and continuing education. 

o Quality educational opportunities must be affordable, convenient, and geographically acces- 
sible for all students. 

o Open educational access requires the use of a variety of instructional strategies and technolo- 
gies to accommodate individual learner needs and varied learning styles. 

o Quality and excellence occur in a continuous improvement climate that recognizes emerging 
technologies, values applied experiences, and advances community and business partnerships. 

o Students deserve a respectful, safe, and caring environment that supports personal growth and 
embraces diversity. 

o Student life enriches, educational, career, cultural, recreational, and social development in 
preparation for citizenship in a global society. 

o Student services enhance educational experiences, promote personal well-being, and support 
student success. 

o Public investment in higher education assumes a promise of fiscal responsibility. 

o Higher education in a multicultural setting values academic freedom, develops critical thinking, 
and accepts philosophicaLdifferences. 



Innovative Designs Planning Stage 3: Formation of Design Criteria 



o 
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The third stage was the establishment of design criteria. Design criteria are features that must be present within 
programs, departments, and services in order to respond to trends and challenges facing RCTC; some might call them 
aims or goals. The vision, mission, values, and market trend analysis resulted in more than 120 ideas deemed important 

BEST COPY AVAILABLE 16? 









Chapter 8. New Designs in Higher Education. Quality Improvement in Higher Education/ 165 



by our Futures Task Force. Using quality tools, the original 120 design criteria ideas were narrowed to 18 and then 
further synthesized to three critical design criteria that will be embedded into all institutional assessment and planning 
activities. The Rochester Community and Technical College Design Criteria are: 

1. The mission and vision are modeled in the delivery and support of teaching and learning. 

2. The college aligns resources to support the mission and vision. 

3. The performance of the college demonstrates continuous improvement. 

To set performance standards for each design criterion, rubrics were created. Join us as we share the "chocolate 
tasting session model” that helped faculty and staff grasp rubric creation. Each design criterion, then, has a set of 
performance standards comprised of four components: a benchmark or standard, desired results, elements of 
performance that must be completed to achieve the desired result [design criteria), and a clear description of. 
acceptable performance. 



Innovative Designs Planning Stage 4: Establishment of a Desired Culture 

The fourth stage in the Innovative Designs process is to establish a statement of desired culture that defines the 
essential characteristics needed to live our vision, mission, and strategic goals. Once again, the Futures Task Force 
utilized quality processes and tools that resulted in the establishment of a statement of desired culture. Because 
culture has as much impact as does strategy in the attainment of strategic goals, a significant effort focused on the 
creation of RCTC’s desired culture. 

□ Statement of Desired Culture 

Rochester Community and Technical College will create a learning community and work culture characterized by 
the following traits: 

o Respecting... the differences and values of all stakeholders 
o Celebrating... our successes/having fun 
o Treating. . .students and stakeholders to their delight 
o Collaborating... through open communication, encouraging innovation 

These characteristics and behaviors are essential to strategic success. 



Innovative Designs Planning Stages 5 and 6: Identification of Performance Indicators 
and Strategy Deployment 

The fifth and sixth stages of Innovative Designs are the identification of key performance indicators and the 
development of strategic deployment. The college has identified five major indicator clusters: teaching and learning, 
satisfaction, partnerships, continuous improvement, and resources. At this writing, these five indicator clusters are 
in an early draft format. Key performance indicators, strategic priorities, and strategic deployment are next steps to 
identify in Stages 5 and 6. Eventually, tied to the six stages and the resultant strategic plan, each area of the college 
will submit annual work plans, including performance indicators, priorities, and strategies that support RCTC’s goals. 
A web-based information dissemination approach will be implemented, providing for rapid access to information and 
alignment of work throughout the college. At the April NCA meeting, we intend to share progress on Stages 5 and 
6 of our Innovative Designs Plan. 



A Systems and Process Approach 

Supporting these Innovative Designs planning stages is a system and process approach to managing the college. To 
facilitate the goal of continuous improvement, RCTC has identified nine major systems that function in the college. 
At the center of these systems is teaching and learning, the heart, soul, and reason for our existence. The eight other 
systems support the design and delivery of teaching and learning. Process and functional teams for each of the nine 
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functional areas are currently identifying and 
documenting key processes and indicators. 
Upon completion of the Innovative Designs 
Strategic Plan, systems and processes will be 
reviewed for alignment to RCTC's strategic 
goals. 



Performance Assessment and 
Management 

The combination of vision, mission, values, 
design criteria, rubrics, statement of desired 
culture, key performance indicators, and stra- 
tegic priorities create a report card or bal- 
anced scorecard to identify what is important 
and what will be measured to determine suc- 
cess. Combined with the Malcolm Baldrige 
Performance Excellence Framework, this pro- 
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Student Life 




Image & Enrollment 
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Teaching and Learning 
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cess provides a structure and approach to review and assess organizational performance, align the organization, 
manage systems and processes, and continuously improve. Six layers of assessment will provide feedback for RCTC 
to continuously improve. For each dimension, design criteria will provide a 
consistent link for review and improved performance. During the 2000- 
2001 academic year, the focus of assessment has been on program review. 



Innovative Designs 




Participation in the Minnesota Assessment and 
Quality Awards Program 

Vital to our journey has been the support of colleagues both formally and 
informally. Active participation in CQIN, identification of NCA Colloquia part- 
ners, and the supportand feedback of the Minnesota Council forQuality have 
helped keep us enthusiastic. The Minnesota Council for Quality sponsors 
the Minnesota Quality Awards, a noncompetitive Minnesota Assessment 
Program. In 1999, RCTC was the only private or public Minnesota 
business or organization to participate in the Minnesota Assessment Dimensions of Assessment 
Program. As a participant, RCTC administered the Baldrige Express 

Survey to staff. The survey feedback results were supplemented by a campus site visit conducted by a team of quality 
evaluators who also provided an extensive feedback report based on Baldrige-d riven Minnesota Quality Awards criteria. 



Course 1 - Outcomes 



Stuff Development 



NCA Reaccreditation 

This past fall, RCTC once again participated in the Minnesota Quality Awards. RCTC's self-assessment used in the 
awards process also serves as the primary document of our NCA self-study. In April, a team of NCA consultant- 
evaluators, with a background in the Baldrige criteria, will conduct our reaccreditation on-site visit. 

For additional information on RCTC’s Innovative Designs initiative contact Dave Weber at [507) 285-7217 or 
dave.weber@roch.edu. 



Dave Weber is Director of Marketing at Rochester Community and Technical College in Rochester, Minnesota . 
Don Supella is President of Rochester Community and Technical College in Rochester, Minnesota. 
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Library Improvements at 
Samford University: 

Listening to the Voice of the Customer 



James C. Eck 
Archie Lockamy III 



Introduction 

During the early 1 990s, Samford University’s quality initiative encouraged the university to seek the Malcolm Baldrige 
National Quality Award. As part of that award process, Samford University collected data regarding its evidence of 
quality. Although the university did not receive the Malcolm Baldrige National Quality Award, the process was 
beneficial to the university. Opportunities for improvement still exist, however, and we offer a few insights about how 
Total Quality Management (TQM] led to higher levels of effectiveness in terms of library services. 

Samford University embarked upon its own quality mission beginning in 1989 with a series of discussions to explore 
what the elements of TQM were and how those elements might be brought into the education arena at Samford 
University. About that time, they hired Dr. John Harris as Associate Provost for Quality Assessment to provide focus 
and direct this effort. In April 1991 a Quality Improvement Conference was hosted by and held at Samford University. 
In attendance were many educational professionals from other colleges and universities as well as several guest 
speakers in the quality improvement field. 

Dr. Corts talked about hearing staff or faculty talk about “if we had money, etc” we could achieve thus and so, for 
example. What he said about quality management is that it removes the money element and allows people to focus 
on where they are nowand what they can do about it. “Excelling” is easy if you have money to fix everything. Quality 
management philosophy allows one to look at “better-ness” in increments. True quality management and improve- 
ment is not just a “flash and crash" project; it is not a one-time program to talk about and flaunt, one that has great 
public relations appeal and then is over. Rather it is a continuous improvement effort in all the things we do. 

An overriding theme of the conference was that a quality management initiative should be handled with “gentleness.” 
It requires a culture change in many organizations, and it takes some employees a while to see the benefits of the 
quality process. 



Opportunity Recognition 

Samford University’s commitment to Total Quality Management now spans over a decade. Samford University’s 
adoption of TQM is as comprehensive as any other university in the United States. The TQM effort has led Samford 
University to emphasize those traits that make this university distinctive. Samford University is distinctive in that our 
tuition is half the national average for private institutions, yet we fund three professional schools (law, pharmacy, and 
divinity] that are more typical of larger universities that enroll more than 4500 students. Samford University is also 
distinctive in that we collect a copious amount of data from our students and we act when we become aware of their 
concerns. Students’ desire for better science labs, for example, resulted in the university’s constructing a new science 
building that cost more than $25 million. Samford University also wants to review student outcome data in order to 
monitor the extent to which the university is providing a quality education and the degree to which students are 
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satisfied with their experience. The data suggest that there are good reasons for Samford University to feel affirmed 
in terms of the quality of education being provided to our students. 

In addition to specific student out- 
comes, Samford University surveys 
students regarding their impres- 
sions of services offered on cam- 
pus. Since 1992, the Davis Library 
at Samford University has adopted 
a strong commitment to TQM. Stu- 
dent satisfaction with library facili- 
ties and services has increased 
each year [with the exception of 
1997-1998, when the library was 
undergoing extensive renovation). 



Improvement Steps 

These improvements are attributable to the leadership of the library’s director and her staff. Ten years ago, library staff 
members were frustrated and demoralized. Consultants evaluated the state of the library, and four librarians 
[including the director) received quality training. Then, a planning team was formed and the library staff members 
were surveyed in order to get their comments on needed improvements. The results of the survey enabled the library 
to focus on one-time actions, possible organizational changes, and ways to improve the library facility. Survey 
responses indicated that the staff wanted the staff room repainted and that everybody wanted to wear name tags. 
The library survey also suggested that there were staffing problems within departments (e.g., too may individuals in 
one area and too few in another) and that similar functions were being performed in several areas. Because of these 
comments, the library formed a reorganization team to develop a new organizational chart for the library. 

The team was successful in its reorganization efforts (e.g., a 21 percent increase in hours that librarians staff the 
reference desk, a 64.3 percent increase in hours that support staff work in circulation) in large part because the team 
met four goals/objectives: back everything up with facts, avoid feelings and emotional reactions, strive to reach 
consensus, and allow the director to have final authority. The most recent renovation of the library (including the 
creation of a student lounge, additional study rooms, an enlarged collection, computer connections at every table, 
and a more attractive interior) has resulted in the highest satisfaction levels among students. The library is a clear 
example of using a TQM approach for moving a student service from a marginal status (58 percent student satisfaction 
rate) to an exalted status (over 90 percent student satisfaction rate). 




Benefits 

As a result of library renovations, students now enjoy the following benefits: 

o Study carrels and reading rooms are equipped with laptops, personal computers, and other resources, 
o All study tables include a computer jack to access Samford’s computer network. 

o The library also houses a cable television network, individual listening and viewing rooms, and a Technology 
in Learning instruction center. 

Student satisfaction surveys show a steady increase in student satisfaction with library facilities and services since 
the 1992-93 academic year. The student satisfaction rate was over 90 percent for the 1999-2000 academic year. 



Recommendations 



Although Samford University’s approach to TQM is comprehensive, opportunities for improvement do exist. First, we 
recommend that Samford publicize the university’s efforts in the quality arena in orderto increase its competitive edge 
and distinctiveness. Many faculty and students are unaware of Samford University’s commitment to quality, and the 
university must adequately inform all of its stakeholders. Samford University needs to adopt additional measures of 
quality (especially qualitative measures) and incorporate opinions from all customers (metro students, professional 
students, graduate students, faculty, staff, etc.). Samford University must incorporate opinions from external 
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customers [e.g., the views of local businessmen/women concerning skills desired among MBA graduates] and 
broaden its level of communication about quality both internally and externally. 



Conclusion 

Taken as a whole, these data suggest that customer focus and satisfaction is a point of distinction at Samford 
University. The customer’s perception of the quality of service that an institution extends is a major factor in every 
operational organization. How an institution commits itself to satisfying customers is similarly vital to the success of 
any educational institution, especially for the quality of services that it implements into its own educational services. 

Downsizing, retrenchment, and doing more with less have become themes for most institutions of higher education. 
Productivity now has a sense of urgency as institution after institution struggles with shrinking resources. TQM has 
allowed Samford University to avoid problems associated with declining enrollments and budget shortfalls. Samford 
University’s mission is to nurture persons, for God, for learning, forever. Underlying the institution’s approach to 
achieving its stated purpose is a concept that should be the primary focus of all colleges and universities-quality. 



James C. Eck is Director of Institutional Research at Samford University in Birmingham, Alabama. 

Archie Lockamy III is Professor of Operations Management at Samford University in Birmingham, Alabama. 
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A Two-Year Technical College 
Approach to Delivering 
Quality Workforce Training 



Jan Cota 
Teri Bradel 
Mary Eaton 



Learning Outcomes 

Upon completion, session participants will have an overview of how the ISO quality standard, a traditionally 
manufacturing-based system, was applied by a two-year technical college providing workforce-training services. 
Participants will learn firsthand the benefits, costs, and “best practices” as a result of implementing a quality standard 
through presentation, simulation, and discussion. 



Introduction: NTC CTS/ED Background Information 

Custom Training Services and Economic Development [CTS/ED] is the division of NorthwestTechnical College (NTC) 
charged with providing a nontraditional external delivery system of consulting and educational services. These are 
provided to business, government, and nonprofit organizations. NTC, through the CTS/ED division, seeks to: 

o Establish strategic economic development alliances at regional and state levels 
o Maintain and strengthen customer relationships 
o Deliver quality workforce and organizational programs 

o Continuously seek new business and the development of industry relationships. 

In an effort to continuously improve services and products delivered to business clients, CTS/ED took the initiative 
to investigate options available for implementing a quality system. 



Selection of ISO as the CTS/ED Quality Standard 

The method chosen by CTS/ED to analyze and improve our system and control the quality of products and services 
provided is the ISO 9000 standard. ISO provides CTS/ED with a vehicle to assure customers of our reliability and 
competence to deliver our products and services. 

The purpose of ISO is to facilitate international commerce by providing a single set of standards that people 
everywhere would recognize and respect. International Standards Organization is the globally recognized quality 
standard used by more than 25,000 businesses in 90 countries worldwide. Located in Switzerland and established 
in 1947 to develop common international standards, its members come from over 90 countries. 

In general, ISO assumes that a wide variety of elements influence product quality. The ISO 9001 Standard has 20 
elements and requires control of the characteristics that make up these elements. Areas covered by the elements 
•nclude items such as: 
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o Management leadership 
o Planning 
o Operations 
o Organization 
o Records 
o Design 
o Training 
o Purchasing 

ISO provides an opportunity to support NCA accreditation of Northwest Technical College programs and offerings 

by providing: 

o Credibility - quality model that is recognized by our customers and business and industry internationally 
o Integrity - quality focus aligned with our mission statement 
o Structure - model for documented processes and procedures 
o Evidence - model for decision by fact [records). 



Implementation of the ISO Standard 

Implementation of the ISO standard has resulted in a defined Quality Policy Statement as follows: 

Northwest Technical College’s Custom Training Services and Economic Development [CTS/ED) Division is 
committed to customer satisfaction by providing products and services that consistently meet or exceed our 
business and industry customer’s requirements in regard to quality of product, quality of service, and overall 
value in an environment of continuous improvement. 

Critical to the implementation of the ISO standard is: 

o Management leadership and participation 
o Assign system director 
o Contract consultant/coach 

o Contract intranet specialist [paper or electronic decision) 

o Training 

- Training orientation 

- Writing 

- Internal auditing 

- Root cause analysis 

- Proposal development 

- Needs assessment/performance outcome identification 

- Hands-on online 

o Establish writing teams-all management level, including V.P. 
o Internal audit participation by all management levels, including V.P. 
o Monthly agenda time-documentation draft presentations and training 



Benefits/Challenges/Investment 



o 

ERIC 



Our ISO experience has given us a unique perspective since we are a service organization and not involved in 
manufacturing processes. Consequently, we have learned the dojs an^d don’ts of applying the ISO standard in a 
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nonmanufacturing setting. We have experienced firsthand the benefits and costs in financial and human terms in 
implementing a quality system standard. 

o Client benefits 

1 . Demonstrate to clients that we “walk the talk” 

2. Established strategic planning goals aligned with quality policy statement 

3. Established client needs assessments: “How will we know if we succeed?” 

4. Established client proposals & contracts with performance outcomes 

5. Improved client response time: system efficiencies 

6. Consistent product quality: product consistency and method to address quality issues 
o Internal benefits 

1. Clarified staff roles and responsibilities 

2. Documented policies, processes, and procedures 

3. Established systematic method for continuous improvement 

4. Single access to controlled documentation 

5. Integrated business management system 

6. Time and system efficiencies 

o Challenges/solution * 

1. Interpreting standard and applying to our business/consultant-coach 

2. Time: assign project leader and establish regular agenda time 

3. Establishing system and documentation boundaries/learn from experience 

4. Establishing value to all internal stakeholders/establish regular agenda time 

5. Communication in multi-site organization/use of intranet 

6. Human impact and ownership/training, regular involvement 
o Investment 

1. Consultant 

2. Intranet specialist 

3. System director 

4. Registrar 

5. Time 

Conclusion: Lessons Learned 

1 . Quality is a process, not an event. 

2. Quality does not require “perfect” to get started. 

3. Maintain buy-in and ownership by providing needed resources. 



Jan Cota is Systems Director, Custom Training Services/Economic Development, at Northwest Technical College in 
Bemidji, Minnesota. 

TeriBradel is Interim Assessment Specialist, Custom Training Services/Economic Development, at Northwest Technical 
College in Bemidji, Minnesota. 

Mary Eaton is Vice President, Custom Training Services/Economic Development at Northwest Technical College in 
D emidji, Minnesota. 
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Mid-Cycle Assessment Review: 
A Mechanism for Creating a 
Positive Assessment Culture 



Jessica L. Jonson 
Thomas C. Calhoun 



Introduction 

In “Proclaiming and Sustaining Excellence: Assessment as a Faculty Role” [1998), Schilling and Schilling “suggest 
ways to approach assessment that are more congenial to the traditional faculty role.” Their suggestions are important 
because they can facilitate the changes needed to create a culture of assessment-that is, a culture where assessment 
evidence is tied to decision making and leads to improvement of educational programs. Our institution has attempted 
to adopt these approaches through a process we call the Mid-Cycle Assessment Review. The purpose of the review 
is two-fold: 

1 . Reflection and suggestions: Provides a forum where a focused discussion about assessment activities allows 
faculty to reflect on their assessment efforts and obtain suggestions or ideas for improving their assessment 
plans so they can gain meaningful and useful information about student learning. 

2. Communication: Provides a mechanism for: [a) clarifying the expectations for assessment efforts, (b) sharing 
successful assessment strategies used by other colleges/programs at UNL or other institutions, and (c) 
obtaining a broad sense of where the institution stands in its assessment efforts. 

The need for this process is illustrated by a review team's report in a recent academic program review: 

The Department’s procedures for assessing student learning in the major are remarkably thorough, but whether 
the benefits to the Department and its teaching mission outweigh the very obvious costs is not clear. In 
discussing the results of last year’s assessment with the Undergraduate Chair and his committee, we were not 
convinced that the attempt to quantify student learning by scoring student performance in a number of inevitably 
subjective categories had yielded meaningful results. In contrast, the exit survey the Department administered 
to outgoing seniors apparently provided many helpful insights into the undergraduate experience. The 
committee [peer review team] suggests that the Department search for ways to simplify its assessment 
instruments so that the costs in faculty time are not so great, and it urges the administration to accept a range 
of assessment procedures that reflect realistic possibilities for measuring student outcomes. 

The comments of the review team reflect the frustration of those faculty who believe that the costs of assessment 
outweigh its benefits. Recognizing thatthe program review serves multiple purposes, we chose to deal with this aspect 
of faculty frustration outside of the program review process by developing a mid-cycle assessment review that occurs 
two to three years before the program review. During the mid-cycle review, the University-wide Assessment 
Coordinator studies the department’s assessment plan; reads its reports; and meets with the faculty to address issues 
of assessment objectives, design, and methodology. We anticipate that this process, given time, will assist faculty in 
developing useful and reliable assessment measures. 

A second concern raised by the review team’s comments is their lack of familiarity with either assessment design or 
university and college policies regarding assessment. Although they suggest assessment simplification and echo the 
faculty’s frustration, they offer no guidance as to howfaculty might simplify assessment and obtain meaningful results. 
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Their suggestion that the administration be urged to accept a range of assessment procedures points out the review 
team’s lack of familiarity with university and college guidelines that already accept a broad range of assessment 
procedures. And, they appear to be unaware that the assessment plan was designed by the faculty, not imposed on 
them by administration. Recognizing that few program review teams will have either interest or expertise in 
assessment design and that many will have only nominal familiarity with university policies, the mid-cycle review 
process was developed to focus on issues of assessment design. The program review team can then be expected to 
focus on the issue of program quality. 



Background 

The preceding example from a program review team's report illustrates several of the reasons why our institution’s 
University-wide Assessment Committee created the mid-cycle assessment review process. A document drafted by 
the committee to establish the review process, fully explains its reasons: 

The office of the Vice Chancellor for Academic Affairs has a responsibility to ensure that guidelines relating to 
assessment information have been followed by each unit in its academic program review. Given that the 
effectiveness of UNL’s assessment plan hinges upon assessment being successfully integrated into the APR 
[academic program review] and accreditation processes, having a way to encourage and monitor this integration 
is needed. The following plan assumes that units have implemented assessment plans having characteristics 
described by the university plan and that they have documented their assessment activities in annual reports 
to their deans. 

Depending upon accreditation standards and the training and interests of external review teams, there may be 
no mention in a team's report of the assessment efforts of the faculty, which can convey the impression that such 
efforts are not valued. This plan ensures that at some point in the program review process, assessment activities 
are focused upon. In addition, employing explicit criteria makes it clear that the university has standards that 
should serve as goals for programs as they develop and refine their assessment plans. Nevertheless, despite this 
degree of standardization, faculty are left great latitude in determining the objectives to be measured and how 
they measure the achievement. 

With responsibility for assessment activities changing frequently, annual reports alone are unlikely to convey the 
broader picture of how or whether assessment is contributing to the growth of a program. Instituting a formal 
assessment review is intended to encourage reflection upon the cumulative effects of the assessment process. 

The mid-cycle review emphasizes the university's commitment to a process of outcomes assessment that 
provides the information necessary for formative program evaluation . 1 

This mid-cycle review process complements the institution’s annual reporting process in which deans are asked to 
forward each department’s assessment report or a summary of assessment activities [program and college level). 
Using this information as a basis for discussion, the mid-cycle review process then provides an opportunity for the 
University-wide Assessment Coordinator to: (1) interact with faculty responsible for the assessment process within 
a college or program, (2) offer suggestions as to how assessment activities might be improved, or (3) gain insights 
to share with other programs and colleges. A five-year rotation schedule that will result in a review being conducted 
with all of the institution’s academic units was implemented in fall 1998. 

Following the completion of the first mid-cycle reviews, the University-wide Assessment Steering Committee revised 
the review process to provide a complete copy of the mid-cycle review report (ratherthan a summary) to the college 
dean. This change provided the same level of confidentiality as in an academic program review but gave the dean 
more information on assessment activities within a department. A second modification to the process, made at the 
conclusion of the fourth mid-cycle report, was to give the department an opportunity to respond to the mid-cycle 
report. This response enabled departments to correct or clarify information contained in the report and to comment 
on suggestions offered by the University-wide Assessment Coordinator. 



Experiences with the Process 

The extent to which the two purposes of the review, mentioned in the introduction, have been served up to this point 
are discussed below. 



Purpose 1: Reflection and Suggestions 

One goal of the review is to assist a department in determining the usefulness of the assessment process in improving 
~tudent learning. Two criteria are used in determining whether an assessment process is useful. First, how well does 
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the assessment process provide insight into learning issues that interest the department? Second, does the amount 
of faculty time invested in conducting assessment result in the evidence needed to improve the educational process? 
Focusing on these criteria in mid-cycle reviews resulted in the following lessons: 

0 Alternative strategies for directly measuring student learning were explored once it was determined that the 
current method was requiring an inordinate amount of faculty time and was not providing useful evidence 
about student learning. The alternative strategies proposed, would make better use of existing courses and 
course products, rather than administering an additional process beyond students' coursework. The benefit 
in using these alternative strategies is that they would minimize the demand on faculty time and provide useful 
information. 

0 The issues underlying the redesign of a major served as a focus for also redesigning the assessment process 
for that major. Because the new major attempted to improve the “progressivity” of the curriculum, obtaining 
assessment information from students throughout their program On addition to information collected at the 
conclusion of their program) was considered. This additional information would assess student learning 
developmentally as well as provide a mechanism for monitoring distinctions outlined for different course 
levels. 

0 It was suggested that indicators of student learning representing various inputs, processes, and outputs in 
the educational process be linked to improve their usefulness. This would involve determining which inputs 
affected the processes that affected outputs. Linking could assist in determining how modification of one 
indicator led to changes or improvements of other indicators. In addition, linking indicators could assist in 
determining what changes might lead to improvements in student learning and whether additional 
information was needed to obtain more compelling and useful evidence about student learning. 

0 Because only a small number of majors graduated from a program each year, it was difficult to draw accurate 
conclusions about student learning from the assessment evidence. It was suggested that assessment 
evidence be collected every year but analyzed only every few years. This scheduling provided more stable 
evidence on which to base conclusions about student learning. In addition, this change made time available 
for addressing the department’s other contributions to student learning (graduate programs, service courses, 
etc.). 

0 To make the assessment process more manageable, a framework was created to determine which learning 
objectives were already being assessed by products from existing courses and internships. This exercise 
ensured that add-on assessment methods would not be created when methods for assessing objectives 
already existed. 

0 The appendix of the mid-cycle report was expanded to serve as a toolbox addressing the specific assessment 
activities and issues of the Department. This toolbox includes practical how-to resources (book excerpts, 
articles, and checklists); examples from other departments, colleges, or institutions; and frameworks for 
organizing and structuring assessment activities. Although this appendix was customized for each depart- 
ment, several resources proved to be applicable to almost all circumstances (sample senior surveys, 
suggestions for writing objectives, etc.). 



Purpose 2: Communication y 

The mid-cycle review has been an effective forum for clarifying expectations and discussing the benefits of outcomes 
assessment. The coordinator has been told on more than one occasion that the meetings provide faculty with a better 
sense of how and why assessment should be conducted and reduce their anxiety about the process. Specifically, the 
mid-cycle review has provided the opportunity to address common misperceptions. 




In addition to clarifying expectations and correcting misperceptions, the coordinator specifically and continuously 
emphasizes that the purpose of the mid-cycle assessment review is to be helpful and constructive to departmental 
assessment efforts rather than to dictate what the department should be doing. The review does so by encouraging 
and facilitating assessment activities that a department will find beneficial to the program's unique goals. The 
coordinator also emphasizes that the purpose of the review is not to focus on a department’s assessment results or 
determine how well the department’s students are meeting learning objectives but to focus on the department’s 
assessment processes. This focus assists in determining how well the department’s assessment activities are 
providing information on student learning outcomes that the program can use to improve the educational program. 
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The review has also been used to share assessment strategies used by other programs and to encourage and facilitate 
the use of existing data in the assessment of learning outcomes. In this year’s mid-cycle reviews, the sharing of the 
assessment instruments and strategies of other programs were frequently incorporated into the mid-cycle review. 
For example, senior and alumni surveys used in other departments could be shared among very different disciplines 
but still provide useful information about methods for assessing student learning 0.e„ survey item format or content). 
In addition, a method used in the Architecture department, for example, that samples student work of varying 
performance levels could also be used by a variety of departments. This strategy could be universally applied because 
the technique involved sampling the best, worst, and average performance for comparison from one year to the next. 

To illustrate the tone of the mid-cycle review process, a few excerpts from a mid-cycle report written by the University- 
Wide Assessment Coordinator follows: 

The approach of the department in outlining objectives for those courses which are core to the discipline is an 
important and appropriate start for measuring student learning in the major. The initiation by the department 
to inform all. . .instructors of the learning objectives for the courses is a very productive step in standardizing what 
students should achieve once they have completed the course.... 

Although most of your learning objectives appear clear and measurable, discussion with your faculty indicated 
that there are a few objectives that do not directly state the goal of student learning. For example, the learning 
objectives for. . . indicate that students should have the knowledge and ability to “engage in the historical and 
contemporary debates in the discipline” and “express ideas orally.” These two objectives relate to goals for 
student learning, but do not provide the direct expectation that would make measurement of them easier. To 
achieve a clearer statement of the learning expected, these two particular objectives might be changed to say 
students will have the knowledge and ability to “effectively debate an historical or contemporary issue in the 
discipline” and “effectively express an idea orally” so that others gain insights into the issue.. . . 

I commend your department for selecting direct measures that capitalize on information produced in everyday 
course work of key courses. These intact products help eliminate the burden on students and faculty to produce 
additional measures for assessment purposes. However, there are issues that should be considered given that 
your assessment is based on individualized artifacts from courses with different instructors making judgments 
about students’ achievement of the objectives. 



Faculty's Response 

Faculty responses on a survey regarding the mid-cycle assessment review process indicated two things. First, all 
departments that responded shared the suggestions made by the University-wide Assessment Coordinator with other 
faculty in the department. How and with whom the information was shared are illustrated in the following comments: 

Highlights of the report were shared with the department curriculum committee and all departmental faculty in 
a written format These highlights were then discussed briefly at a department meeting. 

The department curriculum/assessment committee has seen copies and the department faculty have received 
copies. There have been discussions of specific issues that the report addressed. 

Second, five of the six departments found that the review assisted in efforts to assess student learning outcomes. The 
following comments represent how the mid-cycle review assisted in these departments. 

[The review] held [us] accountable for reviewing our curriculum and its effectiveness. 

[The review] helped us to see the broader picture between the college and the departments regarding 
assessment. [The review] helped us to look at our assessment plan as incremental rather than “doing it all at 
once.” 

Not only did [the review] prompt us to complete [our] outcomes assessment plans for other academic areas in 
our department, but insight was provided that allowed us to make the previously created instruments more 
succinct. Both prompts were appreciated. 

[The review] prompted the department to think carefully about the next stage of its major assessment program 
and to make firm plans for that stage. The discussion of our plans with [the University-wide Assessment 
Coordinator] helped us clarify purpose and procedures, while validating our general ideas. [The] report and 
resources she sent were helpful in keeping us on track and out of trouble. 
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Conclusion 

In the future, our institution will continue to develop the mid-cycle review into a forum for faculty to discuss the 
usefulness of assessment plans and results, how assessment can address issues of interest in student learning, and 
ways in which assessment efforts can be facilitated by various campus resources. Faculty responses indicate that for 
the most part the mid-cycle assessment review accomplishes its purposes. The hope for this review is that it will 
provide feedback to academic units so they can implement meaningful, useful, and efficient plans for assessment. 
Results from these assessment plans can then be used at the time of Academic Program Review to document program 
strengths while simultaneously gaining information about where to concentrate limited resources effectively. 
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An Institution-Wide Procedure for 
Reviewing and Improving 
Departmental Assessment of 
Student Learning Outcomes 



Dennis Holt 
Christina L. Frazier 



The Challenge 

Communicating institutional expectations and monitoring progress while providing units with the flexibility to design 
and employ assessment processes consistent with their goals and the characteristics of their programs are major 
challenges with a university-wide faculty-owned program focused on assessment to improve learning. At Southeast 
Missouri State University, this is accomplished through unit-based assessment plans and yearly assessment reports 
that are reviewed by the University Assessment Review Committee [UARC). This collegial process, the core of an 
assessment program that was cited by North Central Association evaluators as an institutional strength, is built on 
guidelines set out by Cecilia L. L6pez in Opportunities for Improvement: Advice from Consultant-Evaluators on 
Programs to Assess Student Learning. 



Evolution of the Procedure 

Southeast’s assessment plan, approved by NCA in 1994 without stipulations, was developed by the UARC as a 
continuation of commitments dating from 1967. The UARC— composed of faculty representatives from each college 
and school, the Director of Assessment, the Director of Institutional Research, student development representatives, 
and students-is responsible for the continued implementation and modification of the plan. Initially, the academic 
departments and University Studies [the university’s general education program) developed assessment plans for 
each of their degrees that articulated goals and objectives related to student outcomes, identified methods for 
assessing student achievement on the outcomes, and defined a procedure for involving faculty in reviewing and 
responding to the results of assessment. Departments are expected to use multiple methods of assessment and to 
continually evaluate the success of their assessment programs in terms of the usefulness of the information they 
produce. Recently, Student Development and Enrollment Management developed similar plans for their units. For 
some units, the discussions that generated the departmental goals and objectives were a catalyst for change to align 
the curriculum with the directions they established. 

The methods used vary considerably from department to department. Most include data on the performance of their 
students on the university’s writing exit examination. Recently, the California Critical Thinking Skills Test CCCTST) has 
been administered to upper class students, and some departments are beginning to report these data. Many 
departments incorporate information from external examinations, including the MFAT, MTASCP, CPA, PRAXIS, and 
NCLEX. Several have developed their own assessment instruments, ranging from tests to processes for evaluating 
artifacts from courses. Many plans stress assessment of students’ ability to effectively use the information gained in 
the classroom and demonstrate the skills acquired in the studio and laboratory. The Department of Art invites an 
outside evaluator to review students’ work, while the Department of Music uses a series of faculty-scored juries and 
recitals. Evaluations of experiential learning, such as internships and practica, are included in numerous plans. Some 
departments concentrate on assessing students’ knowledge and skills as they complete the program, while others 1 
determine student progress at multiple points in their course of study. 
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In the first phase of implementation, which lasted about one year, departments submitted plans to the UARC for 
approval, and the committee evaluated them using simple criteria: feasibility, identification of student outcomes, 
acceptability of proposed measures, and process for involving faculty. The committee believed, however, that unless 
there was a method for monitoring the implementation of departmental plans, it would be likely that many 
departments would let their proposed assessment efforts take a back seat to the many other day-to-day demands 
they must face. The long path to a culture of assessment would need to be well-marked with periodic evaluations of 
progress that communicate in simple and direct terms the institution’s expectations for student outcomes 
assessment. 

At about this time, the provost and deans had agreed to institute departmental annual reports summarizing, 
essentially, the state of the department each year. This summary originally was to include only enrollment trends, 
faculty productivity, scholarship, and major departmental initiatives, but it was agreed to add a section on student 
outcomes assessment to this list, where departments were to report the results of their assessment efforts. The first 
round of reports was disappointing. It was clearthat many departmental chairpersons did not understand the UARC’s 
expectations, and formats were highly varied and confusing, even for departments that were known independently 
to have ongoing assessment programs. Though there was a mechanism for periodic reporting of assessment 
activities, a simple format for reporting results and a method of evaluating departmental progress toward a culture 
of assessment was lacking. 



The Review Procedure 

The committee settled on four essential elements of a departmental assessment report, each building on the 
preceding components. The “Plan” section provides the basis for the evaluation of the other elements of the report 
and includes the department’s student learning outcomes objectives, a list of the methods used, with each linked to 
the objectives it assesses, and a description of the procedures, i.e., who does what and when. The "Data” section 
includes the results obtained from the assessment activities. To facilitate a meaningful analysis of the information, 
data from the last five years are included when available. In the "Analysis” section, department members summarize 
the performance of their students with respect to their student learning outcomes objectives and identify strengths 
and areas of concern. The use of the information and insights gained from the Plan, Data, and Analysis sections to 
formulate responses, which are reported in the “Response” section, is the critical element of assessment to improve 
student learning. 

Since the UARC's review is intended to be formative rather than summative, communication of the results of the 
committee’s deliberations to the departments is an essential component of the process. Initially the reports were 
scored as acceptable Cl) or not acceptable CO), and the Director of Assessment and the faculty co-chair of the UARC 
discussed the results with each department chair. In the following year, a rubric, which is readily available to the 
university community, was developed to provide a structure for committee evaluation, a vehicle for communication 
of the evaluations to the university community, and a method to quantify a department’s current status and progress. 
The rubric calls for each of the four sections of the report to be scored from 1 to 4, with 1 indicating that a section 
is undeveloped, 2 that it is developing, 3 that it is established, and 4 that it is exemplary. Committee members record 
their evaluations of each section and specific comments on a scoring form. The entries on the form are used to provide 
each department with specific comments on the strengths of each section as well as the committee’s concerns. The 
scores for the Analysis and Response section are also reported as part of the annual departmental profile. At the 
request of chairpersons or deans, members of the committee meet with departments or college groups to discuss 
assessment in general or specifics of a department’s report. Since each section is evaluated, there is an opportunity 
for a department’s efforts to strengthen a section to be recognized, even if the remainder of the report is still in the 
developing stage. 

The active involvement of the academic deans has been an important component in establishing assessment as part 
of the university’s culture. Each dean receives the scores of all departments in addition to the scores and detailed 
comments for each of the departments in his/her college. The reports and the current status of assessment activities 
are shared with the Council of Deans on a regular basis and have been the subject of beneficial discussions. Individual 
deans have worked in various ways to inspire and help departments improve their assessment activities. 



Results 



This process has fostered an evolution of assessment on campus. The efforts to make this a faculty-owned and 
administratively mandated, but not dictated, process have taken time, and the process has not progressed evenly in 
all departments. In the most recent review cycle, the majority [53 percent) of reports were rated overall as established, 
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and 21 percent achieved exemplary status. However, 21 percent were considered developing, and 5 percent were 
rated undeveloped. Twenty-one percent showed significant improvement over the previous year’s report. One benefit 
of campus-wide faculty service on the UARC is that committee members act as agents of change in their departments. 

The actions of the UARC have changed assessment. For example, many departments had established (33 or exemplary 
(43 assessment reports for their undergraduate programs, but failed to provide any information for their graduate 
degrees. As part of the evolution of the process, and consistent with the wording of the rubrics, the committee scored 
each section that did not include information on the graduate degrees as developing (23. During the next reporting 
cycle, most departments included information on their graduate programs. The Dean qf the School of Graduate 
Studies is building on this new awareness of assessment at the graduate level through the work of a committee of 
the Graduate Council. 

Most importantly, the process has been used by departments to focus learning opportunities to improve student 
learning. For example, many departments report reviewing and making specific modifications to courses or their 
curriculum, including the development of new courses to address identified weaknesses. One department has 
developed a program involving professionals in the area to enhance students’ writing skills, which has resulted in GPA 
increases for 80 percent of the students involved. The Supplemental Instruction in College Algebra project was 
initiated to increase student performance and retention of students in college algebra. The Department of Nursing 
has undertaken a series of actions to strengthen students’ performance on the NCLEX licensing examination, 
resulting in a 15 percent increase in the pass rate. 



Scores 


Plan 


Data 


Analysis 


Response 


1. Undeveloped 


Absence of objectives, 
methods of assess- 
ment, statement of 
procedures for most 
programs 


No data reported for 
most programs 


No analysis and 
evaluation of student 
outcomes 


No action response re- 
ported (action response 
may be decision to 
change or maintain 
program elements3 


2. Developing 


Objectives, methods of 
assessment, procedures 
only partially developed 
for most programs or 
not developed for all 
programs 


Data from assess- 
ments only partially 
reported for most pro- 
grams or not reported 
for all programs 


Some analysis and 
evaluation of student 
outcomes for some 
programs, ora signifi- 
cant number of pro- 
grams with incomplete 
analysis and evaluation 


Some indication of an 
action response for 
some programs; action 
response indicated but 
not clearly connected to 
analysis and evaluation 
of student outcomes 


3. Established 


Objectives, methods of 
assessment, proce- 
dures basically in place 
for all programs 


Basic reporting of data 
forall programs (allow- 
ing for some data not 
to be reported because 
not yet available3 


Someanalysisand evalu- 
ation of student out- 
comes for all programs 


Consideration of action 
response for most 
programs 


4. Exemplary 


Clearly stated objectives, 
methods of assessment, 
proceduresand rationale 
for choice of methods of 
assessment relative to 
objectivesforall programs 


Data clea rly reported for 
all methods of assess- 
ment forall programs 


Clearly developed and 
well-thought-out 
analysis and evaluation 
for all programs 


Clearly stated action 
response indicating 
thoughtful use of data 
for improvement of one 
or more programs 
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Sisyphus and the 
Boulder of Faculty Involvement: 
Successful Methods to 
Increase Faculty Involvement 
in the Assessment Process 



Thomas Beery 
John Fallon 



Overview 

Faculty ownership of the assessment process is essential, yet resistance to assessment implementation is common. 
Too often, faculties commonly view assessment as a time-consuming process that results in the collection of 
meaningless data. Faculty see themselves as outsiders in a process that has been mandated by administrations and 
has been forced on institutions by external organizations. Such a view can prove disastrous to an institution and to 
faculty ownership of the assessment process. Most importantly, lack of faculty buy-in to the assessment process 
results in upinspired documentation of student learning-and student learning is not the place to go through the 
motiqns. 

At Lima Technical College, as at many institutions, the faculty had not readily accepted its role in the assessment 
process aspect of student learning. Even though the college has a well-developed assessment plan that is actively 
supported by the administration, sustaining the faculty’s enthusiasm for and involvement in the assessment process 
had been much like Sisyphus’ attempt to roll the boulder up the hill: some short-term progress made, but few long- 
term results. Additionally, each time the bolder rolled back down the hill, its fall made a greater indentation from which 
to begin the next effort. 

Over the past two years, the college has adopted a series of innovative, successful procedures to increase faculty 
ownership of assessment. These efforts have centered around the realization that, in fact, faculty do not care about 
assessment as it is commonly viewed. However, faculty do care about student learning. Refocusing of the college’s 
assessment initiative away from the assessment process to the improvement of student learning at classroom, 
program, and institutional levels has resulted in a much more active and enthusiastic faculty attitude about 
assessment. 

Reshaping faculty attitudes toward assessment required both global, institutional-level changes of philosophy and 
local, program and classroom initiatives that focused directly on teaching and learning. 



Global Efforts 



□ Reorganization of the Campus Assessment Committee 




Although the campus assessment committee had been a joint committee of faculty and administration, the chief 
center of power and, therefore, the chief controlling force had been the Director of Assessment. Because this 
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Director was so knowledgeable and willing to take on responsibilities, it became extremely easy forthe rest of the 
committee to allow him to make all of the important decisions about assessment. Such a situation only enhanced 
the perception that assessment was a mandated administrative function. 

To change this perception and to form a new reality, the entire assessment process was delegated to a faculty 
committee [with full support of the Vice President for Instruction and the Director of Institutional Research). As 
the process was reworked, so was the faculty attitude toward assessment. Faculty now own the assessment 
process, and this ownership has resulted in renewed enthusiasm and a communal sense of responsibility for 
improving student learning. 

□ Revision of the College Assessment Plan 

The college’s assessment plan is an excellent one, but like any other assessment tool, it requires regular review. 
One of the first initiatives of the reorganized campus assessment committee was to revisit the college’s assessment 
plan in an attempt to create more faculty involvement. The committee discovered an unneeded tier of bureaucracy 
that tended to alienate faculty from the assessment process. The assessment plan was revised to move assessment 
closer to the classroom, making program chairs and their faculty directly responsible for assessment of student 
learning in their programs and individual courses. This revision encouraged faculty involvement and made 
assessment results more meaningful to improving student learning. 

□ Revision of Strategic Planning and the Action Planning Processes 

As part of the effort to integrate assessment into the rubric of the college, connecting assessment to the college’s 
strategic planning process was appropriate. The college’s action planning process distributes funds for special 
tactical initiatives beyond base budget. Planning dollars are highly coveted by all college cost centers. By 
connecting assessment to planning, assessment was brought to the forefront. In order to be eligible for action 
planning dollars, all programs were now required to present a summary of theirassessment activities from the prior 
planning year. This summary focused on the use of assessment data to improve student learning. For the current 
planning cycle, programs were required to propose a measurable plan to improve student learning. During the 
following planning cycle, programs were required to report the results of their plan and consequent follow-up 
actions. 

These changes in the strategic planning and action planning processes resulted in increased faculty interest in 
assessment and assisted in focusing assessment initiatives on outcomes and change ratherthan mere data collection. 
Additionally, tying assessment to the funding process reaffirmed, for faculty, the college’s commitment to document- 
ing improved student learning. 



Local Efforts 

□ Education of Faculty 

Assessment became an issue on our campus after the last NCA site visit in the early 1990 s. Although programs 
had been engaged in assessment since that time, there were still faculty who had basic questions regarding 
assessment. Some faculty were not familiar with the basic elements of assessment (language, terminology, goals, 
etc.). To redress this lack of understanding, a series of informal informational meetings was set up to explain the 
basics of assessment. The meetings, which were not mandatory, presented practical examples to explain the 
assessment process, and examples were given to show how assessment could lead to improved student learning. 
These practical examples were essential as we, by design, shifted discussions away from the mechanics of 
assessment to the end result of improving student learning. Since our faculty values teaching and learning, they 
became more interested in the outcomes of assessment. 

□ Communication 

As a supplement to the educational meetings, the theme of improving student learning was further strengthened 
by two other communication efforts. First, we changed the format and theme of our campus assessment newsletter. 
Instead of focusing on general assessment ideas, we invited faculty to submit teaching and assessment practices 
that had led to improved student learning. The result of the newfocus was astounding. What had been a one-page 
general information newsletter, became a six-page letter filled with successful teaching and assessment ideas. 
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In addition to the newsletter revamping, we also instituted an annual end-of-year Assessment Fair. Staff and faculty 
from various programs volunteered to share assessment procedures, outcomes, and resulting improvements in 
learning with the rest of the faculty in a relaxed, fair- like atmosphere. Again the results were astounding. Although 
attendance was not mandatory, approximately 80 percent of the faculty attended the fair, and the discussions 
continued long after the fair was scheduled to end. 

□ Mentoring 

While the educational meetings and the renewed communications efforts were designed for the faculty at large, 
the final efforts to increase faculty involvement were directed toward individuals. We established a mentoring 
program using faculty who had demonstrated a successful assessment track record. These faculty members made 
themselves available, when asked, to assist other faculty members with assessment issues and processes. The 
results of this effort have been very good as faculty feel free to have questions answered in a nonjudgmental 
environment. 



Conclusions 

The implementation of these global and local practices has increased the faculty's enthusiasm for, involvement in, 
and understanding and appreciation of the college's commitment to improved student learning via assessment. The 
result is that faculty are now concentrating not on the process of assessment, but on improving student learning- 
the proper focus of any assessment program. 
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Adding Depth and Vision to the 
Assessment Process 



Patricia M. Dwyer 
Mark Stern 



Introduction 

Shepherd College had a significant problem, as have many other colleges and universities, with developing a strategy 
that would lead to a college-wide buy-in of assessment. Initially, the college faculty senate was given the charge to 
oversee the development of a college-wide assessment process, but little progress was made under this approach. 
Among the problems concerning the senate was the perception that assessment and evaluation of faculty were tied 
together. Many faculty felt that assessment was already taking place with student evaluation of courses and faculty 
evaluation of students through course grades. In addition, there was a concern that assessment could be used against 
faculty in annual evaluation or promotion and tenure processes. Finally, there was the concern that assessment 
activities consumed valuable faculty time and resources that could otherwise be put to better use. Thus, the meaning 
of authentic assessment of student learning, as well as the nonpunitive use of assessment, had to be communicated 
to the faculty before substantial progress could be made in the development of a college assessment program. Faculty 
had to buy in to the assessment process, and they had to develop an understanding that assessment was valuable 
for the improvement of student learning and teaching. 

Under the auspices of the vice president for academic affairs and with the approval of the faculty senate, a college 

assessment taskforce was formed in the fall of 1998 to initiate and manage an overall campus assessmentplan. Before 

this, individual committees had addressed assessment issues— the assessment committee of the faculty senate, a 
general studies committee-but coordination and communication were critical among all the components of campus 
dealing with assessment. The membership of the task force included representatives from faculty, staff, administra- 
tion, and alumni; the purpose of the group was to lend broad-based support to assessment initiatives, keep all 
segments of campus aware of assessment activities, act as a resource, and help facilitate the assessment process. 
The task force developed a strategy for implementing an assessment process in several stages. 



Phase I: Establishing a Workable Process and Formula for Implementing Assessment 

The first stage included both gaining college-wide understanding of the concept of assessment and the adoption of 
a “formula” in which all departments and programs identified three learning outcomes that would be assessed each 
year. During this initial stage, a campus-wide assessment workshop was held to establish a basic understanding of 
such terms as direct and indirect means of assessment, learning outcomes, and capstone courses. 

One major contribution to the development of a campus assessment plan was a workshop given in October 1 998 by 
James and Karen Nichols, assessment experts with the firm Institutional Effectiveness Associate. They met with 
department leaders and academic support managers and staff to formulate a method for identifying student 
outcomes or administrative objectives. For each plan, assessment coordinators would first establish a link to the 
institutional mission. In the case of academic support units, a unit mission statement was also formulated. Each 
department or unit then established three student outcomes or administrative objectives that would be assessed 
during that particular academic year. For each outcome or objective, the department or support unit would determine 
two ways to assess success in achieving these goals. Groups were encouraged to include different forms of 
assessment-both direct and indirect indicators of learning. A criterion for success was established for each means 
of assessment-in other words, how success would be quantified. Thus, when data were collected from the 
assessment activities, the department or unit would have something to measure its results against. 
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During this first year of formal assessment, all academic support units and departments were expected to create, 
at a minimum, a first-year assessment plan. This plan demonstrated a link to the institutional mission statement 
and usually identified three outcomes that would be assessed that year. Departments and academic support 
units submitted plans to the college assessment coordinator in December; the coordinator then facilitated a 
process in which the task force members reviewed each plan and offered suggestions and/or recommendations 
for better clarity in the proposed unit or department plan. The role of the task force reviewers was not to give 
suggestions about particular outcomes, but rather to make recommendations that would help clarify the plan 
Ce.g., Are three objectives identified? Are criteria for success established? etc.]. In addition, the task force made 
recommendations about varying the means of assessment to ensure that direct indicators of learning were being 
utilized. 

Preliminary plans were returned to departments and academic support units by February 1. During the rest of the 
spring semester, departments and units completed the assessment activities they identified in their plans. Over the 
summer, data were collected, and, in most cases, department and units discussed the results of assessment data and 
revised/maintained program components based on these results before the school year began in August. Completed 
plans (with all assessment results and program review based on results) were submitted to the college assessment 
coordinator and the respective division chair by September 30. Assessment plans also became part of the yearly 
review of each division. 



Phase II: Building the Network Through Grassroots Education 

Although the process for assessmentthroughout campus had been established, it was clear that there was still a great 
deal of resistance and misunderstanding about assessment throughout the campus in general. Thus, there was still 
a critical need to enhance faculty and staff understanding of the meaning and process of assessment. During this 
second phase, the assessment office initiated several faculty and staff development opportunities to develop a better 
understanding of the role assessment can play in improving teaching and learning. Regular meetings were held with 
department chairs and academic support managers to clarify information about the timeline and process for 
assessment plans and reports and to share information that would be helpful for all of the groups. One important 
component of these meetings was communicating among the departments the direct and indirect forms of 
assessment they were using and which strategies were effective. An outgrowth of these discussions were mini- 
workshops that focused on topics that would interest all departments: in spring 1 999, a workshop on the capstone 
experience, and in fall 1999, a workshop explaining “primary trait analysis.” Faculty members who had used these 
strategies effectively in their departmental assessment plans conducted the workshops. In addition, the assessment 
office sponsored a campus-wide workshop in October 1999. This workshop featured Doug Eder, an assessment 
specialist from Southern Illinois University. Three members of the assessment task force also attended the 
Indianapolis Assessment Institute in November 1999. 

Student participation was critical to the success of the assessment program. In spring 1999 and spring 2000, the 
assessment coordinator attended student government meetings to inform students about the CAAP testing process 
Ca state-mandated assessment initiative) and assessment in general. In the task force’s second year of operation, 
a student interested in curriculum issues was asked to serve on the committee. In both 1999 and 2000, articles 
explaining assessment activities were included in the college newspaper, The Picket 

In the spring 2000 semester, the assessment office sponsored the first Student Achievement Day, in which students 
presented projects or papers to interested members of the campus community. The day was a great success, with 
several departments represented through student presentations. Presentations ranged from group projects from a 
Shakespeare class, to science lab experiments, to debate team exhibitions. Another such event is planned for 
spring 2001. Although these presentations were not used directly for program assessment, the fact that the 
assessment office sponsored and organized the event continued to make the connection for the campus between 
assessment and teaching/learning issues. 
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In spring 2000, the college also became involved in more intensive discussions about assessment through the 
campus-wide dissemination of Cecilia Ldpez’s “Levels of Implementation.” The levels themselves, and the discussions 
that resulted from the description of their characteristics, helped the programs and academic departments situate 
themselves within the continuum of implementation described in the new NCA document. Those at level one were 
urged to action; forthose at level two, the concrete description of each level assisted them in seeing the positive impact 
assessment strategies could have. Through these grassroots exchanges, a new understanding of the importance of 
assessment implementation was growing and deepening. 
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Phase III: Shifting the Focus from Assessment to Student Learning 

As departments and units gathered data, faculty and staff began using assessment findings for program review. 
Questions about the program’s curriculum and the actual student learning that was demonstrated compelled 
departments to look more closely at the gaps in student learning and the changes needed in programs to address 
these areas. This juncture of the process was critical; while assessment per se did not always make sense to this 
faculty, they were, on the whole, very committed to teaching excellence and strong academic programs. Thus, in 
our third year, we replaced the campus-wide workshops with departmental retreats. Individual departments met 
with the college directorfor assessment of student learning at a site away from campus to discuss program learning 
goals. This time apart allowed departments to review assessment data for the year, clarify departmental learning 
goals, and examine courses that contribute to the goals. A template was designed to help departments more 
formally articulate program learning goals On line with the institutional mission] and course review. During the 
retreats, departments identified the knowledge base, skills, and values they would want a graduate from their 
program to demonstrate. This then led to a discussion of where in the program these skills and content are taught, 
and how the department could assess these areas. These retreats also provided an opportunity for departments 
to informally discuss issues of teaching and learning. The off-campus site was comfortable, and the meals were 
funded by the assessment office. 

In this same year, eight faculty and staff members attended an assessment conference sponsored by the American 
Association for Higher Education (AAHE] in Charlotte, North Carolina. Members of the team included the vice 
president for student affairs, a faculty member from the math department, the community college assessment 
coordinator, the chair of the general studies committee, a professor in the communications department the president 
of the faculty senate, and the director for assessment of student learning. A campus newsletter was initiated during 
the summer to inform the college community about information gained at the AAHE workshop. The newsletter 
continues throughout the school year to keep everyone informed about assessment activities. In addition, a brochure 
with questions and answers about assessment of student learning was published. 

Through the Office for Assessment of Student Learning, this core group of “assessment experts” started a speaker 
series entitled “Focus on Student Learning." These are breakfast or lunch gatherings in which one person who 
attended the conference presents one idea that he/she learned. The schedule of topics for 2000-2001 included use 
of portfolios for program assessment, student learning styles, ways that academic support units contribute to the 
learning process, learning communities, self-assessment as a teaching/learning strategy, and using focus groups in 
the classroom. In addition to the formal series, the assessment office sponsored a faculty panel discussion about 
portfolio assessment. Representatives from five different departments presented information about portfolio use in 
their programs’ assessment plans. The group also represented a range from those at the initial stages of portfolio use 
to the veterans who had been using portfolio assessment for years. 

During this phase, more effort was made to include students in the assessment process in a way that was meaningful. 
CUp to this point, students were included only by means of surveys or test-taking], A student satisfaction survey was 
administered in fall 2000 and was followed up with focus groups on issues of diversity that were facilitated by an 
outside consultant. This project was cosponsored by the Student Affairs Office and the Office for Assessment of 
Student Learning. A follow-up to the student satisfaction survey will also include a series of campus meetings to 
discuss the results and prioritize the issues that need attention. 



Future Development and Conclusions 



Plans are currently underway to integrate learning communities as an option for students taking general studies 
courses. Two groups will be piloted in fall 2001. Learning communities are formed when a cohort of students sign 
up for two or three linked courses. By sponsoring this initiative, the assessment office continues to underscore the 
connection between assessment and innovative teaching and learning. 

Other innovations include a new funding format for assessment that involves mini-grants for innovative teaching and 
learning strategies. Before this, each department was allocated a certain amount of money for assessment activities. 
Some departments were eager to use money for assessment, but others weren’t. In order to better use the budget 
funds this year, we have given departments a cut-off date to confirm the use of allocated funds. After this date, other 
departments/programs can compete through mini-grants for additional assessment monies. Also this year, the 
mission statement of the college was revised to include a statement about the importance of student learning. The 
latter change reflects the buy-in to the importance of assessment by the College Strategic Planning Committee: 
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the small residential setting of the college creates an environment in which students are able to work closely 
with faculty, staff, and administrators who encourage their intellectual growth, personal fulfillment and 
academic and professional excellence. Student learning is central to the culture of our institution, and finding 
ways to improve student learning is a continuing process, (from the Shepherd College mission statement) 

The following statement of philosophy about assessment, linked to the institutional mission, was also approved by 
the task force in spring 2000: 



Statement of Philosophy on the Assessment of Student Learning 

o Promoting student learning is at the heart of every assessment initiative. Student learning is 
the common goal that drives every department and program. 

o Assessment has the potential to promote an atmosphere of learning, cultural diversity, and 
curricular innovation. 

o Assessment of student learning can build bridges between academic and student affairs, and 

between academic support programs and departments. 

o Assessment results will be used to initiate or build service and academic programs and not 
for any punitive actions. 

(statement approved by the Assessment Task Force, April 24, 2000) 



There is no doubt that the faculty and staff are continuing to be educated about the meaning of authentic assessment. 
There is still debate by some on campus about whether assessment is just a passing fad, or whether it is a fundamental 
part of the educational process. As more departments and academic support units gather assessment data and 
experience the positive impact of assessment on their programs, it is more likely that the depth of the buy-in will 
continue to develop, with assessment becoming embedded within and facilitating the learning process. 



Patricia M. Dwyer is Director for Assessment of Student Learning at Shepherd College in Shepherdstown, West Virginia. 
Mark Stern is Vice President of Academic Affairs at Shepherd College in Shepherdstown, West Virginia. 
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The Day After a 
Ten-Year Reaccreditation: 

Using the Levels of Implementation 
to Refocus and Reconnect Assessment 



Ronald R. Dowe 
Mary Mahony 



What happens after an institution achieves-for the first time in its history-a ten-year reaccreditation? What if this 
affirmation comes fewer than three years after a focused visit on, among other issues, assessment and only two years 
after submission [finally and belatedly) of an assessment plan? Is there elation? Relief? Delight? Bewilderment? 
Complacency? Perhaps the answer is “all of the above.” At Wayne County Community College, faculty faced just this 
set of circumstances last year. The college's assessment steering committee, noting the threats that potential 
complacency might have, found an unlikely ally in the NCA’s newly published chapter reference, “The Levels of 
Implementation." Many other institutions may have also faced or will encounter a similar situation. 



Background 

Over the past ten years, assessment has assumed a major role in the North Central accreditation process. During this 
period, institutions struggled to develop plans to ensure the success of their students. Faculty began to look at the 
teaching/learning process in new ways. Missions, goals, measures, objectives: all were scrutinized and frequently 
revised. Eventually, however, it became necessary to assess the assessments. To aid in this process, North Central 
developed a rubric to aid institutions in their self-analysis: these were made available to a wide audience at the NCA 
Annual Meeting in April 2000, and “The Levels of Implementation: Patterns of Characteristics" has become an 
important tool for evaluating and understanding assessment progress. 

Thus, overthe past year, many institutions have been reevaluating their assessment progress using these levels, which 
provide a very informative overview of an institution’s progress in embedding assessment into the campus culture. 
Using this evaluative process at Wayne County Community has allowed our faculty to focus on where we have been 
and where we plan to go. In addition, it has given our assessment committee a framework to refocus our assessment 
goals. 

Our self-analysis involved several practical ways of utilizing these levels, many of which were described by Cecilia 
L6pez at the April 2000 NCA Meeting. The one that is the major focus of this paper is “measuring institutional progress 
by comparing current characteristics with both those traits which characterized the institution in past years and with 
descriptions of the next higher level on the matrix." 

The major area that this paper examines is "Institutional Culture: Mission and Collective/Shared Values.” Reviewing 
the past problems of our institution is instructive. For many reasons, Wayne County Community College failed to 
recognize the necessity of developing an effective assessment program untill 996. In addition, our multi-campus site- 
based organizational structure, large part-time faculty, and ineffective communication system resulted in a 
diminished sense of the shared values that existed when our college was founded in 1969. Thus, the then newly formed 
assessment committee faced many problems both in clarifying our mission and ensuring that this vision was extended 
throughout the college community.' 
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Although we would not characterize it as such at the time-the “Levels of Implementation” had not yet been 
promulgated-our early efforts eventually managed to place us [somewhere] in level two in the assessment process 
by the time of our 1999 accreditation review. After a focused visit in February 1997, when very strict guidelines and 
requirements for submission of an acceptable assessment plan were imposed by NCA, these were some of the major 
steps that we took. First, we reviewed our mission, and the entire faculty adopted the “Philosophy of General 
Education.” For the first time in the institution’s history, faculty agreed upon a statement of how they believed that 
general education was reflected in the curriculum. This was an enormously important step in ensuring that faculty, 
administrators, and staff shared [we believed] the same goals and objectives. The next step involved developing core 
ability statements. These statements were necessary to operationalize the “Philosophy of General Education.” They 
described expected competencies in communication, mathematical thought, application of the methods of science, 
diversity, and critical thinking, among others. Without such statements, a coherent assessment program would be 
impossible. Once again, faculty discussed and approved these statements in department meetings. 

Next, we developed a course analysis survey to map the core abilities across the curriculum. Essential elements of 
this survey included documentation of each applicable core ability in the syllabus and the identification of varied types 
of assessment. In conjunction with this mapping, faculty assessment coordinators were leading department meetings 
in which department mission statements were developed. Syllabi were revised to reflect the departmental and 
institutional mission and the core abilities. Our institutional vocabulary was changing and assessment, measures, and 
student learning gained a new or renewed focus. We found that assessment afforded an extraordinary opportunity 
to improve collegial discussion and address longstanding issues of poor intra-institutional communication. 



We had Come a Long Way! 

At this point, we went through our accreditation review, receiving a ten-year continuation. Ironically, it was also at 
this point that we realized that we had to face an entirely new set of problems. While everyone at our institution was 
elated, too many faculty and staff had a sense of completion, even complacency. They tended to overlook the fact that 
while our progress had been truly significant, we still had much to accomplish. In fact, we were at the midpoint of 
our journey, not the end. 



The Levels of Implementation 

Fortunately, the newly published “Levels of Implementation” provided a method of review, a way of explaining that 
our philosophy must become a living document, subject to revision and change as both our institution and our insights 
changed. Thus, the assessment coordinators from each department began to review our accomplishments. 

We began this as a group/committee process. After earlier distribution of the “Levels,” faculty assessment 
coordinators, who make up the membership of our assessment steering committee, scored the level that they felt the 
institution had achieved on each of the institutional characteristics. We took these disparate scores and derived an 
average score reflecting the assessment of the faculty coordinators. As we began to compare our progress with the 
statements in the “Levels of Implementation,” we recognized not only that that there was wide variation in scoring 
by the committee members, but also that we were able to focus more clearly on general weaknesses and challenges. 
We identified, for example, weaknesses both in our mapping process and in the interpretation many faculty members 
placed on the core abilities. All too often, faculty took the mapping process as either a challenge or a criticism; many 
wanted to do absolutely everything on the list. Definitions of assessment and descriptions of the measures were too 
vague. Eventually, in one of many assessment committee meetings devoted to this discussion, we realized that flaws 
in our core abilities statements caused these anomalies. In trying to be specific and cover every goal, we had ended 
up with a series of assertions that sounded very good, but in actuality meant little. Several of these statements were 
neither measurable nor reliable. 

We realized that we had begun by attempting to fill avoid when we developed the “Philosophy of General Education.” 
We continued to try to fill that void by attempting to operationalize that declaration. We knew that something was 
missing. In fact, one of the steering committee members said, “Of course we have a philosophy of general education,” 
in response to an assertion by NCA staff that it seemed that our faculty did not have such a philosophy. Our response- 
a statement of philosophy and an enumeration of more specific expectations-was appropriate but slightly off target. 

Although our early efforts suggested that we had a shared set of expectations and values, our product merely 
repackaged our public face about what we normally give our students. Our response had actually skirted a more 
fundamental discussion of purpose and values. 
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Using the “Levels of Implementation” as a means to raise fundamental questions, we have taken the first step to recast 
our core abilities so that they become statements that provide a better and more coherent structure for our curriculum, 
one that reflects the core outcomes that we intend, not the reiteration of narrow and unconnected course objectives. 

We discovered that there was actually another missing element-a shared vision for the college. In this continuing 
discussion, with insights derived from application of the levels of implementation, we also realized that a fundamental 
discussion of the values that undergird our curriculum and our teaching must also occur. 



Shared Values 

This exercise brought us to the question of values, a discussion that we realized should precede any discussion of 
shared vision. In this context, it may be useful to view values as how we intend to live, work, and act in reaching the 
future we describe. A vision may be quite clear and unambiguous; it may be strongly shared by all members of a 
community; it may even be compelling. But without values, a vision may not be worth achieving. The Third Reich was 
based on a very clear shared vision; its values, however, were monstrous. 

How should we behave? How should we act? Value is derived from the Latin valere, to be strong. It has evolved to 
be associated with worthiness and valor. Thus, strong core values give strength and meaning to the vision we would 
describe in a new effort to join assessment and strategic planning. Values are behavioral statements. If we “act” as 
we “should,” what would an outsider see us “doing"? 

The words we use to describe our values reflect our personal mental models. As such, we decided that we shouldn’t 
shy away from trying to identify a core set of values as a beginning place for the discussion of meaning. We began 
with a list of value statements and have proceeded to winnow these down to statements where we believe that we 
have, or should have, congruence. 

Thus, the recent work of the assessment committee has been to address these issues and move our institution toward 
level three, “maturing stages of institutional improvement.” We have adopted several strategies. One important activity 
involved a discussion of our institutional core values. The assessment committee began to explore those ideals that 
underlie our institutional behavior. These are the core values that we hold and that must be shared among students, 
faculty, administration, and trustees. This discussion was complemented by an administrative initiative to provide a 
new vision for the college’s future. Two recent faculty organizations days [where all faculty are paid to participate) 
have been devoted to an analysis of who we are as an institution, what we believe in, and where we are going. 

This has been augmented by a virtual classroom we established using the college’s Blackboard.com® web site. 
[Guests are invited to preview the course at http://citweb2.wccc.edu, choosing “courses,” then "faculty resources.”) 
This course is open to participation by all faculty, both full- and part-time, with the assessment coordinators acting 
as instructors. A variety of discussion threads relating to the development of shared value statements is now ongoing. 

The second step involves the revision our core abilities statements. Instead of what we had discovered were really 
unworkable statements, we decided to develop four m eta -statements. The first three reflect our revised core abilities— 
think, value, and communicate-and the final one alludes to our set of assessment measures, act. Adopting this 
change recasts and clarifies our common general education outcomes and places the definition of specific goals, 
objectives, and measures back into the individual departments and programs. It forces a more accurate reflection of 
exactly what goes on in the teaching/learning process in each area. It helps describe general education in more 
meaningful ways than the traditional discipline- and course-specific statements common to institutions of higher 
education. [For example, “Upon completion of the curriculum, the student will . . . recognize the relations between 
American and world cultures” or “identify the major ideas, events, and geopolitical factors that have shaped the 
contemporary world.”) It enables us to evaluate assessment with far greater accuracy. The assessment committee can 
then, through its faculty coordinators, work to ensure that there is full coordination between the institutional and 
program or departmental mission. Thus, each department and program will be responsible for a far more accurate 
set of objectives and goals, which will allow the student to succeed and which will allow the institution to better 
measure educational effectiveness. 

Another initiative involves the development of a template to connect student achievement in each course with 
success in meeting institutional and departmental core abilities. Including such a template across the institution will 
allow everyone-students, faculty, outside evaluators-to understand our goals and measure our success. 

An additional method of communicating the. institutional mission to all segments of our community is our assessment 
certification. The assessment committee created a checklist for individuals, departments, and the institution itself. 
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This allows instructors to ensure that their courses truly match the institutional guidelines for student success. 
Eventually, the assessment committee will publish a list showing how courses fulfill the institutional mission. 

The checklist is extremely simple, yet enormously helpful for instructors. 



Conclusion 

We are a work in progress. Our achievements are significant, but we have much to accomplish. Our vocabulary has 
been enlarged to include the language of measurement and achievement; our sense of our responsibility for student 
achievement is being far more clearly defined. Still, we have much to accomplish. 

There are some faculty, administrators, and staff who resist the idea of change. However, for those who embrace a 
philosophy of teaching that emphasizes student success and continuous assessment, an abundance of tools, 
workshops, and support have been made available. 

We have come a long way. 



Note 

1. The reader is referred to a paper included in the 2000 Edition of A Collection of Papers on Self-Study and 
Institutional Improvement for more information on both the college’s accreditation history and on how 
assessment became a tool for institutional self-improvement. 
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Appendix 

Wayne County Community College Assessment Certification Checklist 



This is a draft of the checklist that has been designed as a voluntary mechanism for ensuring widespread adoption 
of assessment within teachers' normal practice. 





Faculty Behavior/ Action 


□ 


Identify in the course outline the operational skills/knowledge that students are expected to achieve. 


□ 


Construct tests that accurately reflect both the identified operational goals and the class time invested 
to achieve them. 


□ 


Solicit information on how students are progressing before tests. 


□ 


Share feedback with the class whenever information is solicited. 


□ 


Seek and compile student opinions on the adequacy and quality of tests and assignments. Share 
feedback. 


□ 


Seek data on how effective different classroom approaches work (projects, lectures, group tasks, etc.). 
Share feedback. 


□ 


Adjust teaching style according to student input. 


□ 


Share classroom practices that work with colleagues. 


□ 


Seek data your department/discipline needs to make sound, information-driven decisions. 
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Toward More Innovative Governance: 
A Model for Closing the 
Assessment Loop 



Educational improvement is a neverending loop that continuously cycles through various stages before returning to 
an original point, only to recycle through the stages yet again. Educational improvement begins with developing a 
plan of action, identifying progress of students’ learning, implementing modifications to academic programs, and 
monitoring the effectiveness of the modifications through feedback. From the feedback and monitoring comes 
evaluation, which may lead to change and does lead to more implementation, feedback, and so forth. Needed changes 
in curriculum are discovered in the feedback and monitoring of academic program review (Lbpez, 1 998). Assessment 
is not simply data collected and documented; instead, it is the changes and improvement that are made based on 
assessment data. 

The rationales for this session at the 105 th Annual Meeting of the North Central Association (NCA) are two-fold. One 
is to delineate the growth pattern of a governance system where faculty assumes a leadership role in institution-wide 
assessment. The other rationale is to provide examples of innovations tested at Southeastern Oklahoma State 
University (SOSU). This session will be a time for the presenters to share information about their institution’s efforts 
to effectively assess its academic plans and programs through the Institutional Research and Assessment Committee 
(IRAQ. 



The Assessment Loop 

□ Planning 

To accomplish educational improvement, the Commission on Institutions of Higher Education seeks to ascertain 
that universities follow important Criteria for Accreditation. These criteria require that clearly stated purposes be 
established, appropriate to the mission and human resources of the institution. To determine whether these criteria 
are being met, the Commission looks for evidence of a governance system that provides dependable information 
to the institution’s constituencies and accommodates their involvement in the decision-making processes. This 
shared governance system should allow all constituencies to have a voice in the decision-making of the institution, 
in particular, the planning processes. The Addendum to the NCA Handbook (2000) specifically states that 
assessment procedures should involve a variety of institutional constituencies and provide useful information for 
their planning processes (p. 4). At the 1998 Annual Meeting of the North Central Association, Lbpez provided a 
taxonomy of levels of implementation of the assessment initiative. She suggested that the first level of assessment 
include plans and objectives for conducting assessment activities to measure student learning. 



□ Assessment 

0 First level. The assessment portion of the loop should be viewed as having three levels or sections. According 
to Lopez (1998), assessment evolves at different levels. During these levels, constituencies support 
assessment processes, collect data, and establish assessment cultures. The Commission states that the first 
level of assessment should involve gaining support of the university’s constituencies for collecting data about 
the academic programs, especially support from the administration. The Commission asks evaluation teams 
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to consider the institution’s progress in assessment and to look for evolved or evolving shared understanding 
of the purposes of assessment (Addendum to NCA Handbook, 2000, p. 7). 

0 Second level. Level two of the assessment stage exists when student learning and assessment of student 
academic achievement are valued across the institution, departments, and programs. At this level, some 
academic programs have developed statements of purpose and educational goals that specifically mention 
the department’s focus on improving student learning and the importance attributed to assessing student 
learning as a means to that end (Addendum to NCA Handbook, 2000, p. 8). Faculty have developed measurable 
objectives for each of the program’s educational goals and takes responsibility for ensuring that measures of 
student learning are aligned with the program’s educational goals and measurable objectives (Addendum to 
NCA Handbook, 2000, p. 9). Administration expresses understanding of the characteristics and value of 
assessment and promotes the use of assessment results to effect desired improvements in student learning 
and achievement (Addendum to NCA Handbook, 2000, p. 10). 

0 Third level. While the second level of assessment consists of a feedback process to ensure improvement of 
student learning, the third level involves the creation of a culture focused on student learning. According to 
Ott 0989), organizational culture consists of things such as shared values, beliefs, artifacts, and patterns of 
behavior. It is the unseen and unobservable forces behind seen and observable organizational activities (p. 1 ). 
Kilmann and associates [1 985) state that “culture is to the organization what personality is to the individual— 
a hidden, yet unifying theme that provides meaning, and mobilization” [p. ix). At the third level of assessment, 
improving student learning has become an institutional priority, a way of life (Addendum to NCA Handbook, 
2000, p. 8). Institutional statements include language indicating the high value the institution places on student 
learning [p. 8). Faculty speak publicly and informally to their peers and the institution’s other constituents in 
support of the assessment program and educate others about its value Cp. 9). Syllabi for each course state 
measurable objectives for student learning and provide for the assessment of student’s academic achieve- 
ment Cp. 11). The institution publicly and regularly celebrates demonstrated student learning, performance, 
and achievement [p. 13). 

□ Modification 

Closing the loop actually begins with the modification phase of assessment During this phase, changes in 
curriculum are a result of assessment data and are linked to academic program review [L6pez, 1 998). An important 
activity at this level is that effective feedback loops exist so that information about assessment results and 
attempted changes are shared with all institutional constituencies and are used to improve student learning 
( Addendum to NCA Handbook, 2000, p. 12). Another important activity that should occur at this level is that 
administration arranges for awards and public recognition to individuals and academic units making noteworthy 
progress (Addendum to NCA Handbook, 2000, p. 10). Most importantly, closing the loop means improved student 
learning. Williams and Shakiban (2000) argue that the key question of performance assessment is: 

“Can a student, toward the end of any degree program, demonstrate, through production (e.g., capstone paper, 
exhibition, presentation, or exam, etc.) the level of knowledge, skills, and values that is necessary to be 
considered a competent well-educated graduate in that program?” The asking and answering of this question 
should be a core component of an assessment plan for a major, (p. 180 ) 

□ Evaluation 

The final phase of the assessment loop is the evaluation phase, which directly leads to or recycles back to the 
planning stage. Mangrum and Mangrum (2000) contend that monitoring the effectiveness of the decision-making 
and implementation process is usually the phase of assessment that is neglected (p. 20). According to their 
research on shared governance models, a weakness exists at this level. For example, they discovered from a case 
study that structures for monitoring progress and decisions made by committees did not exist. While reports from 
one committee are passed to another (e.g., the assessment committee provides reports to the faculty senate), 
specific methods for monitoring the effectiveness of the shared governance process were not outlined by the 
university. Gollattscheck (1 985) recommends involving all constituencies in constant monitoring in terms of where 
recommendations go, who handles them, and who is effective at it. 



Action Plan to "Close the Loop" 

This section of the paper provides a summary of interventions taken by the Institutional Research and Assessment 
Committee (I RAC) at Southeastern Oklahoma State University to overcome its assessment problems. This committee, 
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consisting mostly of academicians, carries the university’s primary responsibilities for all academic assessment 
programs and program review functions. The activities used by IRAC to engage the constituencies of SOSU include 
revising reporting documents, conferences, workshop retreat, and rewards for assessment reports. Before describing 
these activities, the next section will provide a brief history of assessment problems at SOSU. A rationale for the 
interventions as well as descriptions of them will then be outlined. 

□ Assessment Problems of SOSU 

Southeastern Oklahoma State University is a regional university with primary emphasis on quality undergraduate 
education. The university’s objectives are to provide programs of instruction in the arts and sciences, business, 
education and behavioral sciences, and technology that lead to baccalaureate degrees. The growth pattern of 
governance where faculty assumes a leadership role in institution-wide assessment has evolved dramatically in 
the last few years for SOSU. This evolution has come about for three reasons. The first reason described by Weiner 
and McMillan C2000) entails “changes that [are] necessary to move forward and meet the NCA mandate for 
shared-governance” [p. 300). The second reason was a change in administrative officers, with the current 
administration supporting faculty involvement in governance. The third reason is the enthusiasm generated within 
and by IRAC, the responsible committee. No institution is likely to succeed if either of the last two reasons is missing. 

In the past, self-analysis of the problems related to assessment at SOSU has not been focused on academic 
programs, but rather it has been more focused on institutional management. Realizing the significance of this 
problem, the university has engaged in extensive self-analysis to discover weaknesses in academic programs. 
Faculty members found that formal decision processes are usually planned for and described in university 
documents, but in terms of implementation, this was not the case. They found the weakness began with the second 
phase of the assessment loop that consists of steps to implement the decisions made in the planning phase. For 
example, in the Self-Study Report 0998) written by SOSU, there is a flow chart explaining the steps of decision 
processes by listing inputs into the process, such as faculty senate charges, faculty initiatives, administrative 
requests, outputs such as administrative actions, faculty senate recommendations, and committee modifications. 
Of the ten levels of actions described in the flow chart, only the last level addresses implementation, and it simply 
states "implementation occurs exit process.” (A copy of this flow chart will be available at the presentation and in 
the Resource Room of the 2001 NCA Annual Meeting). 

Another problem addressed by IRAC concerned what shaped many departments’ view of assessment-just 
another duty to be completed, checked-off, and ignored until the next due date. This problem is related to the third 
level of the assessment phase. The IRAC committee found that the institution does not show evidence of having 
a culture where faculty and administration engage in conversation about ways to use results of assessment efforts, 
and, in particular, they are not focused on improvements in student learning. Feedback from committees such as 
IRAC was filtered through the administrative process and lacked immediacy (and thus lacked effectiveness). This 
in turn led to a mismatch of feedback with subsequent reports and perhaps indirectly encouraged apathy about 
assessment measures and reporting. Therefore, the departments conducted little follow-up with the information 
discovered by the program assessments. 

□ Interventions 
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This following section of this paper focuses on several intervention projects conducted by IRAC. With changes in 
the administrative perspective of governance, the committee was allowed to create innovative changes through 
interventions. It developed several interventions as a way of changing from ineffective methods of accomplishing 
assessment to a better way in hopes of closing the loop. The purpose of the innovations and interventions was to 
identify barriers to program assessment and to encourage departments to implement changes based on the 
assessment reports. 

The major issues of this session are growth and innovation. IRAC has been moving from working in a reactive mode 
to working in a proactive mode. It has moved from simply responding to administrative requests-checking-off one 
more item on the list-to addressing assessment problems, brainstorming, and looking for ways to improve student 
learning and the university’s effectiveness. For Southeastern Oklahoma State University, the IRAC committee 
serves as a hub for the network of diverse programs and departments to promote the sharing of institutional 
assessment strategies. This committee is required to be the catalyst for growth and innovation. To accomplish these 
goals, IRAC began several interventions to assist the university in creating an assessment culture. These 
interventions include 0) revising reporting requirements, C2) engaging in conferences, C3) conducting a workshop, 
and C4) awarding effective assessment programs. The following sections describe each of the intervention 
strategies. 
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0 Intervention #1: Revised reporting requirements. Even though the institution has officially been 
developing the assessment process for the last ten to fifteen years, the majority of changes have been made 
in the last four years. Three years ago, the role of the oversight committee was revised and elevated (Weiner 
and McMillan, 2000, p. 301]. Two years ago, among other tasks, the committee reviewed and revised remedial 
education requirements and compared reports to plans. Last year, the committee began the process of 
comparing the reports with plans for continuity. However, it quickly became apparent that there were 
problems with the process. The committee realized that it needed to develop a new form with which to evaluate 
whether the department was attempting to close the loop. This involved comparing the report with the plan 
and then looking for evidence of growth or change in the departmental report. The form simply asked if these 
changes were evident in the departmental report. It also asked if changes had been made and if the changes 
had led to improvement in student learning. [A copy of this form will be available at the presentation and in 
the Resource Room of the 2001 NCA Annual Meeting]. 

Many of the departmental reports failed to show that any such feedback was taking place. However, there 
were probable explanations for this problem. The administrative form used to request a departmental report 
of progress on the plan was deficient. This form contained discrete spaces for answers. Departmental chairs 
and others who completed the form often filled the spaces and no more. Both the spacing and the requested 
information failed to allow for the feedback needed to close the loop on assessment. Some departments 
demonstrated knowledge of the entire process from plan to feedback to evaluation adjustment and so forth. 
However, the majority of reports were incomplete matches with the related plans. Either those who filed the 
report responded only to the exact request, or filled the space, mentally checked-off the task and ignored the 
purpose of the report. IRAC quickly realized that the administrative request needed to be revised. The ex officio 
administrator agreed and worked to change the request form. (A copy of this form also will be available at the 
presentation and in the Resource Room of the 2001 NCA Annual Meeting]. 

0 Intervention #2: Conferences. In the 1999-2000 school year, IRAC also addressed another problem. 
Feedback from IRAC was filtered through the administrative process and lacked immediacy. To intervene, the 
committee conducted conferences with key people to identify barriers to program assessment, and to 
encourage them to implement changes based on the assessment reports. The chair of IRAC worked in 
cooperation with all campus academic deans to arrange a brief meeting with each department chair to discuss 
the developments in assessment campus-wide and to discuss the departmental report. Participants at the 
meetings included the respective departmental chair, the dean, the administrative ex officio member of IRAC, 
the IRAC chair, and the IRAC reviewers of the departmental report The IRAC chair explained that these 
meetings were being held to share communication between departments and the committee. Meetings were 
also held to share, in general terms, what the committee had learned about the state of assessment campus- 
wide; to shorten the time from submission of report to feedback from IRAC; and to offer suggestions [or to 
receive them] that could assist the department. The ex officio member of IRAC discussed, in general terms, 
some qualitative measures that could be used to expand assessment. These meetings seemed to be viewed 
as helpful and worthwhile. 

0 Intervention #3: Workshop. Another engaging intervention project sponsored by IRAC included an all-day 
university- wide workshop held in an off-campus location. Lunch was provided to all participants. Events for 
the workshop include a presentation by a representative from the North Central Association, faculty from 
academic departments that submitted top program assessment reports, and activities designed by organiza- 
tional communication and training and development professors from SOSU along with other members of 
IRAC. This workshop will be conducted before the presentation in April, so details will be provided in the 
presentation or in the Resource Room at 2001 NCA Annual Meeting. 

0 Intervention #4: Award for top assessment report in a school or college. Administration provided an 
additional amount of discretionary funding for the program reports that were judged by IRAC as the way to 
close the assessment loop. The department that received the award will share ideas with other departments 
during the Intervention Workshop. More details of these awards also will be provided at the 2001 NCA Annual 
Meeting. 
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Sustaining a Culture of 
Assessment Via Collaboration 



Luis "Tony" Baez 
Evonne Carter 
Don Weimer 
Vicki Martin 



Student outcomes assessment has been a major initiative in colleges across the country over the last decade. 
Institutions receiving accreditation from NCA have been diligent in implementing outcomes assessment not only to 
satisfy the NCA requirements, but also to focus on student learning outcomes and to increase accountability to their 
communities. In 1998, Milwaukee Area Technical College (MATC) had its ten-year accreditation visit and was granted 
continued accreditation by NCA with the next comprehensive evaluation in ten years. However, a focused visit on 
student outcomes assessment (SOA) was required, primarily because of concerns about the breadth of buy-in 
throughout the college. Immediately following the news of the required focus visit, MATC’s administration and the 
SOA committee met to strategize and “regroup.” The committee was charged with realigning its efforts and devising 
a plan to shift the onus of responsibility from the SOA Committee to the entire college community. Thus began a two- 
year effort to more closely link the SOA initiatives to administrative activities, to the efforts of Institutional Planning 
and Quality, and to the expertise of the Institutional Research personnel. This collaboration has resulted in notable 
progress and an improved “culture of assessment” at MATC. 

Milwaukee Area Technical College is a large, urban, two-year technical college with more than 60,000 students 
enrolled each year. There are four campuses across a two-county area of Southeastern Wisconsin. There are 107 
associate degree programs offered, as well as College Parallel courses. Faculty number about 1 300: 600 full time and 
700 part time. 

MATC has supported the assessment initiative in a number of ways. College-wide committees have been formed for 
Strategic Planning, Curriculum and Learning, and Student Outcomes Assessment. Release time for curriculum work 
has been made available; college-wide assessment days have been planned; faculty institutes have been developed; 
and opportunities for travel and conference attendance have also been made available, as has clerical support for 
revising courses. 



The Student Outcomes Assessment Plan 

MATC completed its first Student Outcomes Assessment Plan in 1994. Over the last six years, the plan has evolved 
and grown due to the efforts of the SOA Committee, changes in the nation’s view of assessment, and as a result of 
administrative support within the College. MATC’s initial efforts of outcomes assessment were focused on program 
outcomes and course-level learning. Although these foci have not been eliminated, the current plan is much more 
encompassing. The plan is cyclic and three-tiered: assessment at the course level, at the program level, and for the 
College’s Core Abilities. 

Course-level assessment is done as summative and formative evaluations of learning and to improve instruction. 
Program-level assessment focuses on broader learning outcomes as a culminating process for improvement of 
program offerings. The assessment of the General Education Outcomes/Core Abilities is conducted not only in the 
Liberal Arts areas, but also as part of each occupational program. Faculty members are responsible for assessment 
in each of these areas, sometimes as individuals and sometimes as colleagues in a program or a department. Multiple 
forms of assessment are used, depending on the data sought for specified outcomes. 
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Course-level assessment has two purposes: (1) to determine a “grade” for a course, and (2) to provide information 
that can lead to improved learning by determining which students have attained the stated expectations. A major 
change has been the college’s endorsement of a competency-based curriculum model. Part of the assessment plan 
calls for the revision of all the college’s 2000 courses into this format. 

For each program at MATC, learning outcomes are being assessed with the intent of making improvements in the 
program, which will lead to enhanced student learning. After the outcomes have been established and assessed, the 
data collected are then analyzed, leading to decisions about learning, teaching, course sequencing, curriculum, and 
delivery. The outcomes assessment process for programs has been called “PT A 3 ’’-which stands for Plan, Teach, 
Assess, Analyze, and Adjust (based on the Continuous Quality Improvement Cycle that has been adopted by MATC). 
SOA is an ongoing process that requires faculty to be involved in all the steps and activities, especially in the decision 
making activities that lead to changes in the curriculum and in classroom activities. It is through data analysis that 
faculty members can pinpoint areas of strength as well as areas that need improvement for program level learning. 

To support the centrality of general education at MATC, the Core Abilities of the college are an integral component 
of each course, and student demonstration of these is determined through assessment in all programs. As with 
assessment of course competencies and program learning outcomes, numerous assessment tools are implemented. 



Collaboration Efforts Leading to an Improved Culture of Assessment 

An organization is more than a sum of its parts: it continuously grows and changes, and it requires symbiotic 
relationships among its parts. This interdependence and collaboration need to be cultivated. By linking and aligning 
the efforts of the Administration, the College’s Core Committees (Student Outcomes Assessment, Curriculum and 
Learning, Strategic and Quality Planning Steering Committee, and Minority Student Participation and Retention 
Committee), Institutional Research, and Institutional Planning/Quality, MATC has been able make significant and 
supported changes in its effort to sustain a culture of assessment. 

0 Administrative efforts. Student Outcomes Assessment began as an initiative that was to be faculty-driven. 
Setting goals and outcomes, identifying and implementing assessment tools, analyzing the data from the tools 
and adjusting teaching and learning activities were logical steps that faculty members could take. However, 
the process is labor intensive, and making changes often requires administrative leadership and input. Initially, 
the SOA process was not readily taken on by the faculty. In fact, many seasoned instructors were hoping to 
wait out this new idea. Adding on another layer of assessment was not deemed beneficial by some of the 
people whose reality is the day-to-day interaction with students, not program outcomes or core abilities. The 
true wake-up call was the news of an impending focused visit on SOA. A directive from the college President 
and Vice President of Academic Affairs confirmed the fact that the faculty-driven initiative had to be supported 
and led by all administrators. 

0 Quality/CQI. The Center for Continuous Quality Improvement has been in existence since 1 992 at MATC. The 
role of the Director has been to train college personnel on quality practices, provide facilitators to improve 
meetings and processes, and coordinate core committee activities. In addition, the Director provides guidance 
on effective strategic and quality planning models and practices. The strategic and operational planning 
processes have consolidated to include budgeting, human resources, technology, and institutional effective- 
ness to assure alignment and avoid duplication of effort. 

In the Strategic Plan, Student Outcomes Assessment objectives have been a critical part of the “Focus on 
Learning” Goal for the College. Since it is part of the long-range plan of the college, it is recognized as a key 
component of the future success of the college. 

0 Support from the Department of Institutional Assessment, Research, and Development (IARD) Just 
as assessment is a natural activity of instructors, evaluation is an activity of IARD. Typically, the efforts of these 
two entities ran parallel at MATC. As the SOA plan evolved, it was deemed necessary for the expertise of the 
people in IARD to benefit the activities of the faculty. Conversely, the types of evaluations done by IARD on 
programs should include faculty efforts in assessment and delivery. In 2000, the College hired a Research 
Manager with extensive experience in planning and assessment. With the additional resources, IARD was able 
to provide a renewed level of technical support to SOA activities. Among this support was the College’s first 
systematic attempt to assess student achievement in the core abilities. 

The Committee had established indicators or behaviors that show the attainment of these core abilities in 1 999. The 
Committee then developed a rating instrument using a Likert-type scale and asked faculty to rate students and 
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students to self-rate on the frequency with which students exhibited these behaviors. Two smaller programs were 
selected from each of the College’s academic divisions for the pilot. Several measures were compared, including: 

1 . Comparison of faculty to student ratings overall and by program 

2. Comparison between program faculty, to examine inter-rater reliability 

3. Coefficient of reliability internal to the instrument [Cronbach’s Alpha) 

4. Ratings of the performance on the indicators themselves, overall and by program. 

Preliminary conclusions suggest that the project was successful. Numerical measures were most heartening, 
especially considering that it was a pilot project. A group of faculty analyzed the feedback based on the pilot and made 
several recommendations to modify the instrument and the process, which were incorporated into a second 
assessment in spring semester 2000. Results are not available as of this writing. 



Another area of support is a blind assessment of course-level assessment of the core abilities. Rubrics had been 
selected by department committees to analyze students’ work to determine how the core abilities were taught and 
learned at the course level. In the fall 2000 semester, a double-blind evaluation system of student work was 
established, based on a random selection. This support has and will significantly streamline this process. 

This year, the SOA Coordinator and the Research Manager have set out to determine the degree to which actual 
changes have occurred due to the efforts of the last six years. Just as the plan and its implementation are complex, 
so too is this evaluation. We are attempting to collect data to answer the following questions: 

o To what extent does the organizational climate support quality, specifically for SOA? 
o How engaged are the programs and departments in the SOA process? 
o How are data collected and analyzed? 

o What is the level and quality of collaboration in programs and departments for SOA? 
o How are improvements identified and implemented based on data collection processes? 
o How does the college support these change efforts? 
o What are the long-range implications of the SOA efforts? 

Throughout the school year, data have been collected from faculty, administrators, and students in the form of surveys, 
focus groups, interviews, and document analysis [assessmenttemplates, divisional budgets, and minutes of meetings, 
for example). The analysis of these data is a priority for the research department, the Director of Planning, the 
SOA Committee, and the Vice President of Academic Affairs. As the culture of assessment at MATC continues, the 
efforts of the entire college community are pooled and focused on continuous improvement. 



Luis ‘Tony” Baez is Vice President of Academic Affairs at Milwaukee Area Technical College in Wisconsin. 
Evonne Carter is Student Outcomes Assessment Coordinator at Milwaukee Area Technical College in Wisconsin. 
Don Weimer is Director of Institutional Research at Milwaukee Area Technical College in Wisconsin. 

Vicki Martin is Director of Strategic and Quality Planning at Milwaukee Area Technical College in Wisconsin. 
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Appendix 



Sustaining a Culture of Assessment Via Collaboration 



Goal 



Developing and Sustaining a 
Culture That Supports 
Student Outcomes Assessment 



Involvement 



Administration 

Increased accountability 
Administrative evaluations 
Divisional teams 
Departmental meetings 
Clerical support 



Strategic Planning and Quality 

Stakeholders’ meetings 
Planning goals 
College-wide reporting 
Semester updates 
Links to budgets 



Student Outcomes Assessment 
Committee and Coordinator 

Focus on building and enhancing a 
culture of assessment 

Improved communications 
Staff development 



Institutional Research 

Assistance in developing tools 
and analyzing data 

Data collection and dispersion 
Evaluation of assessment program 
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Twenty-Eight Right- and 
Left-Handed Tools for All Seasons: 
A Toolbox of Useful Items to 
Embed a Culture of Assessment in 
General Education, Discipline Level, 
and the Classroom 



Russell J. Watson 
Jan A. Geesaman 
Peter T. Klassen 
Ingrid L. Peternel 

This paper, even as the presentation, will address a variety of tools that can be used in any institution to assist in its 
assessment efforts. College of DuPage has used many different approaches from a broad base to impact assessment 
at classroom, discipline, and general education levels. In addition, we have attempted to complete the feedback loop 
throughout the process. This paper will be divided into four segments, even as the toolbox presentation is divided. 
Each segment wilj develop a different layer of the College of DuPage assessment process. 



Classroom Assessment 




Tool #7; Our injtial focus developed a system of reporting and documenting classroom assessment activity by 
our full- and part-time professors by using a feedback form that is submitted to the Classroom 
Assessment Coordinator, then published both in hard-copy as our “Linkages" document and also on 
our web site. (Forms are distributed as a part of the toolbox package for the presentation.] 

Tool # 2 : Classroom assessment has been the topic of a faculty in-service day, and agenda as well as Power Point 
slides are presented as a part of the toolbox documentation. 

Tool #3; Our Teaching and Learning Center, a faculty resource system, has provided 50 copies of the Cross and 
Angelo book, Classroom Assessment Techniques, to be checked out by faculty and used as their own 
resource tool. 

Tool #4: Each academic discipline has an Assessment Liaison person to help answer questions and provide 
resources for discipline members who have questions about classroom assessment issues. 

Tool #5: In order to bring faculty up to speed in assessment processes, we wrote a faculty-development class 
on Classroom Assessment Techniques. The class was offered for semester-hour credit and included 
activities, assignments, and demonstration of assessment projects by the faculty participants. [Agenda 
for the class is included in the toolbox.] 
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Discipline-Level Assessment 

Tool # 6 : At the discipline level, we began a system of assessment in multi-section courses in several areas. This 
involved faculty developing an agreed-upon set of assessment options, comparing data, and compiling 
a summary form to submit to the Discipline-Level Assessment Coordinator. These summaries are also 
published in hard-copy and on the web site. (Copies of a variety of discipline report forms are supplied 
in the toolbox.) 

Tool #7: College of DuPage has an internal mini-grant system under the umbrella of the Innovation Incubator 
0/0 program. This year the I/I is encouraging projects related to assessment and especially emphasiz- 
ing improving reading skills in our students. 

Tool #8: Several areas have developed a two-purpose tool in a pre-test/post-test design. This model is used in 
two ways: (1) to confirm appropriate placement in the class, and (2) to provide documentation of 
progress. Developmental writing classes and reading classes are successfully using this method, 
among others. 



General Education Assessment 

Tool #9: We implemented a general education assessment process using the ACT/CAAP instruments and a 
unique statistical model that has received national attention. In brief, we assess a random selection of 
entry-level classes each fall, and a random selection of exit-level and end-of-sequence classes each 
spring. Approximately 25 to 30 sections of classes are selected, enough to result in a finished sample 
of about 600 students, or 1 00 students taking one each of the six CAAP batteries. In each classroom 
randomly selected, all six ACT/CAAP batteries are given. Therefore, if an entry-level math class of 30 
students is selected, five students will take the reading test; five students will take the science test; five 
will take mathematics; and five also will take the critical thinking, writing skills, and essay writing tests. 
Through a unique statistical regression and structural equation model (developed by Dr. Peter Klassen), 
we are able to compare scores of entry-level versus exit-level students without matched sets. (Back- 
ground: The Student Outcomes Assessment Committee (SOAC) reviewed available assessments of 
general education. Th rough theSOAC’s review, thefaculty members endorsed theassessmentsastesting 
the things that we taught. This certified the connection of the assessments and instructional processes.) 

Tool #10: A workbook was published to introduce the general education assessment process to the faculty. The 
workbook presented, the general education statement from the college catalog and separated it into 
seven general education outcomes. The general education skills tested in the CAAP tests were 
presented, and faculty were asked to relate these skills to success in their courses. Responses were 
summarized and distributed (on paper and via the web site) back to faculty. Thus the connection of 
general education outcomes with the assessment tools was established in the minds of those who need 
to accept it: the faculty. 

Tool #77: We have published the results of the CAAP testing in two bound documents, also available on the web 
site. The summary documents have been of enormous benefit as we plan strategies for improving 
student success. 

Tool #72: We have also published a colorful initial report on our CAAP results, along with an accompanying 
feedback form for faculty response. The Z-fold document was initially used to get information about 
assessment in front of the faculty in a visible way, and to encourage their response. All responses were 
then posted to our web site, and are there for the reader’s review. 

Tool #13: Afterthe initial results of the general education CAAP assessments, we hosted a series of faculty forums 
to explore and discuss the results in more detail. (For future forums, we plan to entice a few more 
attendees with food offerings such as snacks or vegetable trays, etc.) 



Resources for Faculty, Students, and the Community / Academy At Large 

Tool #14: In order to get broad-based interest and support, a staff development workshop was designed with an 
outside speaker (JeffSeybert) to help kick off our assessment efforts. The initial workshop went a long way 
in getting the original idea out and in promoting the assessment initiative, especially at the classroom level. 
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Tool #15: Funding is always an issue, and usually begins on a shoestring. Our initial funding came by soliciting 
seed money from other committees to cover expenses for the initial CAAP testing. Once the value of 
that part of the initiative was established, funding gradually increased, resulting in line-item position 
within the college budget 

Tool #16: The development of a specific assessment web site was of enormous benefit to the college and the 
greater academy. The web site serves as a clearinghouse for information, feedback forms, papers, links 
to other web sites, and current news about assessment at College of DuPage. (The site is at 
www.cod.edu/outcomes.] 

Tool #17: Our Assessment Plan had a workbook concept as a part of the roll-out of the plan. That workbook 
design assists in chronicling our assessment efforts, and serves as a guide for new faculty and 
administrators. 

Tool # 18: Motivation of faculty to participate in assessment activity means clear communication with the faculty 
about assessment processes. A newsletter is used for this purpose, published regularly and distributed 
to both full- and part-time faculty. (Copies of the newsletter are also included in the toolbox.) 

Tool #19: Letters to faculty whose classes were randomly selected for ACT/CAAP testing have helped explain the 
purpose, and have helped encourage faculty to participate. At the date of printing this article, about 
150 faculty have been involved, and only one refused to participate. 

Tool #20: Students also need to be encouraged to participate and to do their best on the CAAP tests. A letter is 
addressed to the students involved, and a brochure is distributed to them in class prior to the CAAP 
session. We provide a $20 gift certificate to the college bookstore in a random drawing for every 
twentieth student who submits a valid answer document. In addition, those students who complete a 
valid answer document are given priority registration for the next quarter’s classes. (Copies of the letter 
and student brochure are included in the toolkit.) 

Tool #21: Perhaps the most visible of our tools is an assessment booklet published for the college and the 
community/academy at-large. The booklet describes the assessment process at the college, explains 
how it fits into the college organization, and clearly defines the responsibilities of faculty and 
administrators in the process. 

Tool #22: Our student newspaper, The Courier, has emerged as an important communication vehicle to serve the 
students with assessment-related information. In our first round of CAAP testing, we learned that our 
students’ reading scores were not as high as their skills in other CAAP areas. The Courier had a feature 
article on reading that explored the results of the CAAP tests and interviewed many students for their 
responses. 

Tool #23: Perhaps this is an obvious tool, but we have attempted to engage the maximum number of faculty to 
attend the palette of various assessment conferences offered locally and nationally. This has provided 
a growing broad base of support and expertise in assessment. 

Tool #24: Perhaps this is also an obvious tool, but in the beginning of our assessment efforts, visiting other 
institutions (both locally and a few more distant) was a helpful activity. 

Tool #25: We have encouraged the colleges around our location (DuPage County, Illinois) to develop and 
participate in an annual Assessment Fair. This very local, one-day assessment conference allows for 
presentations similar to those on a national or regional scope to be available on a one-day basis for 
colleges in our area. 



Unused but Useful Tools 

Every toolbox has a tool or two that was purchased because we might need it someday. The following four items are 
in that category at College of DuPage. We think they are useful tools to pull out when needed, but they have not yet 
been tried at our institution. All of the other items have been used successfully. 



Tool #26: An idea of producing buttons or badges that have an assessment logo or message is something that 
we have yet to use, but it may serve an appropriate purpose, perhaps useful for collecting feedback 
forms, or engaging participation in an assessment function. 
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Tool #26a: In the same genre of the buttons, but a bit more effort, is the idea of creating T-shirts with an assessment 
logo, design, or message. These could be given to those who have demonstrated a slightly higher 
commitment, perhaps on the completion of an assessment project or faculty development class. 

Tool #27: The thermometer- or barometer-of-progress in an assessment initiative can be a visible way of 
documenting participation: assessment forms, participation in forums, etc. 

Tool #28: An unused suggestion by a former committee member: An Assessment Party with hors d’oeuvres 
exclusively for those who have completed some portion of an assessment initiative. It was planned as 
a small price to pay to help generate goodwill and upbeat participation with assessment documenta- 
tion, feedback, and follow-up. 

Conclusion 

Assessment is a process-a complex process that involves students, faculty, and administrators in exploring what they 
do, how weii ihey uu il, anu wnat Luey couicl 06 Going cetter as each pursues amplifying student learning success. 
Because it is a complex process, tools need to be developed to address specific needs of constituents along the way. 
It is the intent of the presenters to provide as many tools as possible to help increase the probability of success for 
other institutions. Participants and readers are encouraged to visit our web site, which has many of the tools and 
resources for viewing and downloading Cwww.cod.edu/outcomes). 



Russell J. Watson is Professor of Psychology and Co-Chair of the Student Outcomes Assessment Committee at College 
of DuPage in Glen Ellyn, Illinois. 

Jan A. Geesaman is Associate Dean of Liberal Arts and Co-Chair of the Student Outcomes Assessment Committee at 
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Peter T. Klassen is Professor Emeritus and an assessment consultant in private practice Cp.klassen@worldnet.att.net). 

Ingrid L. Peternel is Associate Professor of Speech, and Classroom Assessment Coordinator at College of DuPage in Glen 
Ellyn, Illinois. 
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New Designs for Linking 
Process to Outcomes: 

The Winona Assessment Project, 

Year Two 



Susan Rickey Hatfield 
Britt Yackey 



The significant problems we have cannot be solved at the same level of thinking with which we created them. 

-Albert Einstein 



In the early 1990s, the North Central Association of Colleges and Schools, responding to increasing demands for 
accountability by the public and legislators, began to require assessment plans for all of their institutions. The 900 
+ colleges and universities of higher education accredited by NCA created assessment plans while the threat of losing 
accreditation hung in the balance. Many university administrators and faculty members comforted themselves with 
the belief that interest in assessment would diminish by the time of their next accreditation visit. Plans created were 
based in the best practice of the time, and largely limited to classroom-level assessment or broad-stroked university- 
level assessment based on existing data, process measures, irregular data collection, and convenience samples. 

It has since become apparent that assessment as demanded by NCA and professional accrediting bodies (not to 
mention the public, granting agencies, and various elected public officials) has evolved into a more sophisticated 
requirement than can be met by most of the assessment plans submitted to NCA in the early 1 990s. This evolution 
in thinking about assessment is evidenced by NCA’s recently developed Levels of Implementation Matrix that outlines 
four criteria against which universities, colleges, and departments can benchmark their progress. The criteria 
identified in the matrix are (1) institutional culture (mission and collective values); (2) shared responsibility (faculty, 
students); (3) institutional support (structures and resources); and (4) efficacy of assessment. The characteristics 
identified in each of three levels of implementation (beginning stages, making progress, and maturing stages) not 
only help colleges or universities identify where they are, but also identify what additional steps or actions need to 
be taken to further their assessment initiatives. 

Unfortunately, NCA’s decade-long assessment initiative has not always resulted in a similar evolution of assessment 
practices on campuses. As it has become apparent that the push for assessment of student learning by the accrediting 
agencies is not going away, colleges and universities may need to reconsider their approach to it. 

The Winona Assessment Project, funded in part by a five-year, $1.36 million U.S. Department of Education Title III 
grant, attempted to overcome the limitations of the current assessment practice on campus by developing an 
integrated database, World Wide Web-based assessment modules, and analytical templates allowing faculty 
anytime-access to assessment data. The goal of the project is to design and implement an integrated database that 
will (1) bring together the existing databases on campus; (2) allow the collection of assessment data in an ongoing, 
systematic manner; (3) create an analytical engine that will facilitate data analysis; (4) provide reports allowing for 
comparative analyses, trend analyses, predictive modeling, etc.; and (5) identify triggers for the development of 
appropriate interventions. The project, now in its second year, started with a rethinking of the current practice in 
assessment and identification of its limitations. 
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Rethinking Assessment 

□ Accreditation 

The current assessment movement on most campuses can be directly attributed to pressure from the six regional 
accreditation agencies. Few universities have comprehensive assessment plans that do not ebb and flow with 
accreditation cycles [though NCA’s AQIP initiative may help change that). 

The pressure of mandated assessment is a mixed blessing. On the one hand, it has forced colleges and universities 
to take assessment of student learning seriously. On the other hand, accreditation mandates can promote a sterile, 
mechanistic approach to assessment that generates little enthusiasm or excitement. Assessment can become a 
bland exercise in filling out forms and checking off requirements instead of a process that engages a university’s 
collective creativity and imagination. Assessment programs that consist of bursts of assessment activity in the two 
years of the self-study are usually followed by a collective sigh of relief— and a return to business as usual— following 
the site team visit. There is no ongoing, systematic improvement as result of the data collected as assessment is 
the means to the end. not the beginning of the future. 

In order to be effective, assessment needs to focus on improvement, not accreditation. Assessment will capture 
the energy of a campus only if it is seen as a genuine, sincere effort on the part of a university to improve itself 
through continual analysis, discussion, and innovation. Changes made as the result of assessment data need to 
be public, publicized, and assessed. Without a clear feedback loop and focus on institutional improvement, 
assessment will be a continuous uphill battle from which no one emerges better for the effort. 

□ Leadership 

Assessment must be championed by the highest administrative officers on campus. Resources and support are 
essential to making assessment work, not just for collecting the data. What is needed is a deeper commitment of 
resources for studying the data and creating change. It is not an inexpensive undertaking. 

While support of assessment at the administrative level is essential, assessment must become faculty driven, not 
administrator driven. Faculty must embrace assessment for the way it can provide information that can improve 
teaching and learning, promoting the scholarship of assessment. This will require faculty to shift focus away from 
thinking about assessment as a remote university exercise in evaluation to the recognition that faculty have the 
power and the responsibility to assess, adapt, and innovate on the classroom, curriculum, and program levels to 
maximize student learning. This is a leap of faith for faculty on many campuses who still harbor the fear that 
assessment data will be used in renewal, tenure, and promotion decisions. It also requires that access to timely, 
relevant data is readily available to faculty. 

□ Implementation 

Universities can utilize a broad range of tools to assess student learning. Commercially or locally developed surveys 
and tests are part of most comprehensive assessment plans. These instruments, when used overtime, provide a 
baseline from which the university can monitor progress or compare itself to national norms. But there is an 
inherent danger in locking into specific tools or methodologies, as they can limit the university’s vision and horizon. 

Assessment must be evolutionary, not method driven. Assessment activities need to be cumulative, building on 
each other, shaped by previously gained knowledge. The menu of assessment techniques and methods that a 
university utilizes needs to be both fluid and dynamic, evolving as data is triangulated to create information that 
forms the hypotheses that are then tested. 

As new questions arise, new assessment techniques and methods are required. Continually measuring the same 
variables the same way may provide evidence of value added, but also may limit understanding of the issue. In 
assessment, there are no wrong answers, only questions that go unanswered because schools are locked into 
methodologies and tools without regard to how the data will shape knowledge. Universities become locked into 

comparing themselves against the past or against other universities instead of positioning themselves to the future 

and the good of their students. 

□ Data Collection 

Many universities have responded to the assessment initiative by concentrating on the collection of data. Huge 
data warehouses have been created, with hundreds if not thousands of data elements. On many campuses, much 
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of the data collected is never used. Colleges and universities need to focus on collecting data that have decision- 
making utility for the campus. 

Institutions undertaking an assessment initiative should start with an examination of every piece of information 
that is currently collected from students and examine each question for its potential utility. If the question is vague, 
if it provides no clear information, if it does not help the university understand its students, if it has no foreseeable 
use as an independent or dependent variable, then collecting it is a waste of time. Many institutions could make 
great strides in their assessment initiatives by streamlining their data collection to include only those variables in 
which they can envision an analysis scenario. 

□ Data Storage 

The problem of an overabundance of data is exacerbated by the inability to efficiently and effectively store data 
in a way that makes complex data analysis possible. At most universities, student data are stored in isolated data 
warehouses or silos, a separate one for each type of data [satisfaction surveys, admission data, graduating senior 
survey, standardized exams, etc.). These silos are often located in different offices throughout the campus 
[Admissions, Registrars Office, Housing Office, Academic Affairs) and have no interconnectivity, as each database 
came online at different times and without universal data collection standards. Additionally, in efforts to maintain 
the integrity of the data by preserving the anonymity of the respondents, data cannot be linked together. 
Relationships between inputs, processes, and results go unexplored. 

Data need to be connected in way that allows for the complex analysis between inputs, process, and outcome 
variables. Assessment is essentially an exploration of the dynamic relationships between data. Isolating data limits 
the ability to learn from the data. 

□ Context 

Current assessment wisdom and practice promote the analysis of student cohorts against each other or against 
external benchmarks [for instance, first-year students’ satisfaction compared to the satisfaction level of seniors, 
or the percentage of students graduating in four years as compared to the system’s or legislatively identified target 
percentage). Though often collected in a systematic and efficient manner, these data are of limited utility in creating 
knowledge because they are removed from the individual student, and their richness has been lost. Assessment 
data need to be interpreted in relationship to student’s goals, not cohorts or mandates. It is the relationship between 
the student and the data that creates knowledge. There is no knowledge inherent in the data themselves; 
knowledge can come only from the interpretation of the data in relation to the goals and background of individual 
students. For instance, rather than measuring an institution’s success based on the percentage of students who 
graduate in four years, a more meaningful measure of success would be the percentage of students who graduated 
in four years who had intended to do so. Many students, because of obligations to their families, jobs, or simply 
personal choice, do not have four-year graduation as a personal goal. Using a static measure such as four-year 
graduation rates is unreasonable unless it is a student’s goal to graduate in that time period. 

Cohort analysis can still be valuable, but it is the student demographic data the define the cohort, not convenience 
of the institution, system, or legislature. 

□ Variables 

An assessment initiative must be grounded in the university’s mission, core values, and key processes. Colleges 
and universities need to measure what they value, and subsequently, value the measures. The danger lies in 
selecting and valuing measures, regardless of whether they are relevant to the core values and key processes of 
the institution. This is especially problematic for institutions using standardized exams and surveys. Though 
standardized instruments allow the opportunity to add items (sometimes as many as 20 or 30), many of the existing 
survey items are irrelevant to the specific institution, or stated too broadly to have any real information value or 
provide any direction to the institution. Though national comparative data are helpful, it is unclear how much of 
the data is genuinely helpful except for occasional publicity purposes. Many institutions could sharpen their 
assessment programs by designing their own instruments, rather than continuing to be both overwhelmed and 
limited by standardized data collection tools. 



The Winona Assessment Project 

To date, Winona State has identified the key variables needed to measure the achievement of the mission, core values, 
and key processes of the university. The variables have been identified in the existing databases and pulled into a 
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separate read-only database that is updated bi-weekly. Online student assessment modules have been developed 
to measure key processes and are currently in the second stage of testing. Four analysis tools have been developed. 
Faculty and staff training on one of these tools has begun, with the other tools continuing to be tested and validated 
for training of faculty and staff late in the spring 2001. The assessment project will feature prominently in Winona 
State’s evaluation visit in September 2001. 



This project merges Winona State’s assessment vision with a state-of-the-art technological solution. The anticipated 
result is a significant shift away from thinking about assessment as a periodic, isolated event precipitated by 
impending accreditation visits to a comprehensive, ongoing data collection, analysis, and information tool that will 
revolutionize the way Winona State practices assessment and facilities student success 



Susan Rickey Hatfield is Assessment Coordinator at Winona State University in Winona, Minnesota. 
Britt Yackey is CEO of Destination Knowledge in Chicago. 



Summary 
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Merging Assessment with 
Everyday Academia: 

A Universal Model That Works 



James D. Evans 

It is NCA’s ideal to have each of its member schools reach a level of assessment maturity that makes assessment 
procedures “a part of everyday life” (Lbpez, 2000). Many colleges and universities are exhibiting steady progress 
toward that goal, but others are at an impasse for two reasons: 

1. There is a troublesome perception that assessment requires faculty members to use out-of-the-ordinary 
measures-such as standardized tests-that cannot be easily worked into normal classroom activities. 

2. Among NCA member schools, there is a clear preference for assessment methods that measure specific skills, 
processes, and competencies. Yet the majority of professors find it difficult to see how they might gauge 
student achievement along such dimensions (Lind, 2000). Simply put, they don’t have enough measurement 
experience to translate their usual evaluation procedures into competencies assessments. 

This paper (1) presents a universal method of assessment that uses the professor’s normal course exams (and other 
evaluation techniques) as the foundation of the assessment program; (2) describes how to combine Howard 
Gardner’s Multiple Intelligences with Benjamin Bloom’s Taxonomy of Cognitive Processes to form a comprehensive 
assessment system; (3) describes specific techniques for implementing this assessment system within a number of 
different methods of testing and grading students; and (4) responds to the most frequent faculty objections to 
quantitative assessment of specific competencies within the context of college courses (see Knox and Knuesel,2000). 
The NCA Annual Meeting session corresponding to this paper will provide empirical examples of the application of 
this system at a mid-sized university; describe how the system has been used to improve course delivery; and outline 
a plan for training an entire faculty in the implementation of this approach. 



The Course Profile Concept 

The competencies-oriented assessment device I wish to suggest is built upon a combination of Bloom’s (1956) six 
cognitive operations-Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation-and Gardner’s 
0 993) expressive modalities, as shown in Competencies Matrix 1 . Gardner identified eight expressive modalities that 
he considers to be distinctly different “intelligences”: 




Linguistic: 
Musical: 
Mathematical-logical: 
Spatial reasoning: 

Movement or bodily-kinesthetic: 
Interpersonal: 
Intrapersonal: 
Naturalist: 



use of words to express knowledge and reasoning 
expression through sensitivity to pitch and rhythm 
use of symbols and formal logic in reasoning 

expression of knowledge and reasoning concerning the relative positions and 
orientations of objects in space 

psychomotor conceptualization and expression 

expression of interpersonal sensitivity and inference 

sensitivity to one’s own emotions, moods, and motives; self-understanding 

ability to identify and classify patterns in nature 
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A complete college curriculum includes learning and training along most of these expressive dimensions. When these 
dimensions are combined in all possible ways with Bloom’s six cognitive skills, it is not difficult to conceive of a formal 
assessment model that reflects the great diversity of interests and outcomes normally observed in a college-student 
population. Thus, each cell of Matrix 1 represents the combination of a particular competency and a particular mode 
of expressing that competency. 

Competencies Matrices 2 and 3 illustrate how the “expressive-modality/competency” profiles of typical courses might 
differ from one course to another. In practice, any college course can be profiled in such a fashion. "High,” "Medium,” 
and “Low” stand for high, medium, and low emphasis of the modality/competency combination. A blank cell means 
that the particular combination is not substantially present in the course. 

Under the assessment system described here: 

o 

o 
o 
o 



Every general education course and major core course is profiled within this expressive-modalities/ 
competencies matrix, using the course’s final exam, final project, or final product as a course-profiling guide. 

Every year, each of the profiled courses is assessed forthe level of student achievement in each of the modality/ 
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The level of student achievement in each of the modality/competency areas of medium and high emphasis 
will be tracked from year to year. 

Results of student-performance tracking will lead to changes in teaching strategies to increase the level of 
student achievement in modality/competency areas that are deficient-that is, that show lower-than- 
acceptable achievement. 



Competencies Matrix 1 



Expressive Modality 


Competency 




Knowledge 


Comprehension 


Application 


Analysis 


Synthesis 


Evaluation 


Other 


Linguistic 
















Musical 
















Mathematical-Logical 
















Spatial 
















Bodily-Kinesthetic 
















Interpersonal 
















Intrapersonal 
















Naturalist 
















Other 

















Competencies Matrix 2: Example of a Course Profile for a Typical College Course 




Expressive Modality 


Competency 




Knowledge 


Comprehension 


Application 


Analysis 


Synthesis 


Evaluation 


Other 


Linguistic 


Medium 


High 


High 


Low 


Medium 


Low 




Musical 
















Mathematical-Logical 


Medium 


Medium 


Medium 










Spatial 
















Bodily-Kinesthetic 
















Interpersonal 






High 


Medium 


Medium 






Intrapersonal 






High 










Naturalist 
















Other 
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Competencies Matrix 3: Second Example of a Course Profile for a Course Emphasizing Higher Skills 



Expressive Modality 


Competency 




Knowledge 


Comprehension 


Application 


Analysis 


Synthesis 


Evaluation 


Other 


Linguistic 


Medium 


Medium 


Medium 


High 


High 


Low 




Musical 
















Mathematical-Logical 


Low 


Medium 


High 


High 


High 


High 




Spatial 
















Bodily-Kinesthetic 
















Interpersonal 


Low 


Medium 


Medium 


High 


High 






Intrapersonal 


Low 




Medium 


High 








Naturalist 
















Other 

















As will be made evident later in this paper, the assessment system discussed here has several distinct advantages 
over the systems used at many colleges and universities: 

o It is extremely easy to conduct assessment analyses using this system, because the methodology asks 
professors to work with procedures and information they ordinarily use in their classes. 

o This approach has content validity because it links assessment directly to course learning objectives, target 
skills and knowledge, and relevant within-course evaluation procedures. 

o The universal ease of implementation means that a school that uses this approach will have a uniform and 
meaningful cross-curriculum method for carrying out a comprehensive assessment program. 

o Many of the assessment devices employed in higher education either yield only very general outcome data- 
for example, “32 percent of our students made As in their senior-level courses” (what does that tell us?)-or 
tend to be based on subjective impressions or anecdotes. In contrast, the methodology described here yields 
quantitative indicators of the degree to which students can demonstrate specific skills and processes in pivotal 
courses. 



Implementing the Device 

This approach to assessment can be implemented most conveniently in courses that involve comprehensive final 
exams. However, creative professors will also be able to make this system work well with semi-comprehensive finals 
(covering the second half of the course), the combination of a unit final with one or two other unit exams, projects 
and performances, or research reports and term papers. 

□ Start with the Final Exam 

Let’s take the most straightforward situation. Assume that an instructor administers a comprehensive final test 
assessing the students’ mastery of the most important course material and competencies. Of course, the primary 
purposes of such an exam are to ascertain the degree to which the students have learned the subject matter and 
to help determine what grades they should receive. But, for assessment purposes, there is much more that one 
can do with the final. The nature of the exam questions provides information necessary for profiling the 
competencies and modalities considered most important in the course, and the results of the exam gauge the level 
of mastery of these critical skills. 



0 

ERIC 



Why the final exam? In many, if not most, courses, the final exam contains questions that reflect the material and 
skills that the instructor hopes to have taught and developed throughout the entire course. The main assumption 
of this approach is that the final exam questions assess attainment of the most central learning objectives. Indeed, 
the content of the final is often the best indicator of what competencies have been promoted in a course. 
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□ The General Procedure 

The assessment procedure follows these steps: 

1 . Within any course selected for profiling, the instructor analyzes each test question on the final exam (or every 
measurable aspect of a final paper or project] within the framework of the Bloom/Gardner matrix illustrated 
earlier. That is, each question should fit primarily within one cell of the expressive-modalities/competencies 
table. A question (or measurable aspect of a paper or project] that reflects more than one competency or 
expressive modality should be classified according to its most important or salient competency/modality 
combination. Some long-answer essay questions can be subdivided into several cells of the matrix. (See the 
later discussion of this matter.] 

2. If a question (or measurable aspect of a paper or project] has content that emphasizes competencies or 
expressive modalities not included in the table, the instructor can use the “other” dimension provided and 
specify the unique skill or modality. 

3. based on this analysis of question types, Li ie instructor then completes a forma! profiling of the course as 
exemplified by the matrices shown earlier. 

4. The instructor “competency-grades” the exam (or paper or project] to ascertain the percent success rate for 
each competency/expressive-modality combination featured in the course. 

5. Finaly, the instructor prepares a competency-attainment table or chart for assessment-reporting purposes. 
These tables and charts serve as relatively precise devices for year-to-year tracking of student success along 
various dimensions of competency. Their content directly identifies the teaching/learning areas in which 
course-delivery strategies and tactics need to be improved. 

□ Specific Methods 

The particular method necessary for converting students’ performance on exams and papers to “percent success 

rate” will vary from one grading system to another. This section details how several often-used approaches to 

evaluating students can be adapted to the general procedure. 

0 Objective exams. Objective exams include those that use true-false, multiple-choice, matching, and (to a 
lesser degree] completion (fill-in-the-blank] test questions. With such tests, specific sets of items are 
identified with particular cells in the expressive-modalities/competencies matrix. Each set of items is then 
competency-scored by finding the average percent correct in that set through (number of items correct/ 
number of items in the set] x 100. The resulting number is the percent success rate for the corresponding 
modality/competency cell. Suppose, for example, that a multiple-choice test has six questions that measure 
application of verbal concepts. That is, they represent the Linguistic/Application cell. Each student receives 
a percent correct score for those six items. The average percent correct for those six questions across the 
whole class is the percent success rate for application of linguistic concepts in the course. The same test might 
have eight questions that assess logical analysis. That is, they represent the Mathematical-Logical/Analysis 
cell. The class’s average percent correct on that set of items is the percent success rate for logical analysis, 
and so on for each modality/competency combination represented on the test. 

0 Essay exams. Some professors might have trouble determining how to apply this assessment system to essay 
exams, since such tests can be scored in various ways. Also, a long-answer essay question might require the 
student to make major use of more than one modality/competency combination. This section will briefly 
address these issues. 

o Points awarded per question. If the instructor awards a certain number of points relative to a 
predetermined number of possible points for each essay question, then each student’s percent 
correct for that question is simply (points earned/possible points] x 1 00. These are averaged across 
students to get the percent success rate”for the modality/competency cell represented by the 
question. If two or more essay questions fall into the same cell, their percent success rates can be 
averaged to arrive at an overall rate for the modality/competency combination. 

Letter grade per question. Some instructors do not use points to evaluate essay responses. Rather, 
each question gets a letter grade: A, B, C, D, or F. In this case, the instructor should define what he 
or she considers an acceptable grade-let’s suppose that that is a grade of C. In this case, percent 
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success rate of the modality/competency combination represented by the question is the percent of 
the class getting a C or better grade on the question. 

o Pass/fai/ per question. Sometimes an instructor has a pass/fail criterion for each test question, and 
uses neither points nor letter grades to evaluate students’ responses. In such a grading method, the 
percent success rate for the modality/competency cell corresponding to each question is simply the 
percent of students who receive “pass” on the question. 

o Qualitative grading categories. If a professor uses qualitative labels, but not letter grades, to express 
evaluation of an essay response-excellent, good, adequate, inadequate, for example-the item 
assessment situation can be handled in the same way as that described for a letter-grade method. 

o Question of which skill is measured. Most test questions tap more than one competency. For instance, 

an item asking a student to “Compare and contrast the process of operant conditioning and Darwin’s 
survival of the fittest notion” requires the student to exercise basic content knowledge, comprehen- 
sion of concepts, and analysis and synthesis. Logical processes are necessary, but so is linguistic 
expression. How do we categorize such multidimensional test questions within the expressive- 
modalities/competencies matrix? There are at least two ways to handle this, and which solution is 
chosen depends, in part on whether the question is in objective or essay format: 

1. The instructor should ask, “Which skill is most important in the question?” That is, what 
competency was the instructor trying to tap when he or she wrote the question? 
Alternatively, the instructor could ask which competency is likely to be the strongest 
determiner of the student’s success in responding to the question. The answer to one of 
these queries will lead to a reasonable matrix classification of the test item. This solution 
would work especially well for objective exams, but it could also be applied to short-answer 
essay tests. 

2. If the multidimensional test item at issue is a long-answer essay item that prominently taps 
two or more modality/competency combinations, it might be necessary and desirable to 
grade the question in two or more segments. For example, the test item might ask the 
student to “describe and evaluate” something. Separate percent success rates could be 
developed for the skill of basic knowledge acquisition (the describing part) and the skill of 
evaluation, respectively. 

0 Courses with multiple instructors. Different professors teaching the same course might administer 
different final exam questions and have overlapping, but somewhat different, expressive-modalities/ 
competencies profiles. Does this partial differentiation pose a problem for the assessment procedure 
advocated here? Potentially, yes. But the problem can be overcome easily if the professors in question are 
willing to work together on the development of the assessment data. The faculty members start by sharing 
their course profiles based on their respective final exams or projects. For modality/competency cells for which 
both (or all) have data, they would average their percent success rates. Any percent success rates for cells 
unique to particular professors would be reported intact; no averaging would be necessary. 

Another approach to the individuality-of-content problem that is especially applicable to general education 
courses would involve a collaborative effort among all the instructors of a particular course to develop common 
objectives and final exams. The advantage of this option is that it would induce faculty members to think about 
and debate what they think is most important for students to learn. 

0 Papers and journals. Some courses do not require students to take exams. Often a major paper or project 
is assigned and used to evaluate the degree of learning that has taken place. Papers can be graded according 
to many competency criteria, such as accuracy and completeness of content (basic knowledge), abstractness 
of expression (comprehension), thoughtfulness (analysis), organization and clarity of expression (synthesis), 
and critical thinking (evaluation). 

One also finds it easy to imagine assessing journal entries modality/skill combinations such as interpersonal 
analysis, intrapersonal applications, logical evaluation, and so on. 

Likewise, each student in a participation-type class can be classified as below average, average, or above 
average on any modality/skill combination that is emphasized in class discussions and activities. Percent 
success rate is figured on the basis of the percent of the class receiving average or above average ratings on 
each combination. 
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0 Performances. Certain courses in Performing Arts and Communications involve the evaluation of various 
aspects of psychomotor competence as part of the grading process. The student is rated and/or graded on 
knowledge, application, synthesis, and evaluation in the musical, spatial, bodily-kinesthetic, and interpersonal 
modalities. The initial assessment task in such courses is for the instructor to develop an explicit list of the 
modality/competency combinations that he or she considers most prominently in evaluating students’ 
performances. Then it becomes a matter of deciding what constitutes an acceptable level of performance and 
determining the percentage of students reaching this level for each modality/competency cell considered. 



□ Possible Objections to the System 

0 The question of academic freedom. One of the misgivings professors have about competency-based 
assessment is that such an orthodoxy forces them to teach certain competencies. It does not. It merely asks 
professors to identify the competencies and expressive modalities that they choose to emphasize in their 
courses and then assess the degree to which their students exhibit learning along the chosen dimensions. 
This system does assume that we all know what skills and expressive modalities we are trying to develop 
in our students, and that we can identify dirferenitieyiees of these operations in our students’ performance. 
But that is practically a universal assumption in academia. In fact, any argument in behalf of academic 
freedom and responsibility must make this assumption. 

0 Reluctance to replace present assessment methods. Some faculty members will feel threatened by this 
idea, thinking that they already have a good assessment program and that it will be displaced by the new 
system. Replacement isn’t necessary. All that is useful in existing methodologies certainly can be retained, and 
the approach advocated here can initially be used as a more specific supplemental method. I anticipate, 
however, that many disciplines will find the new methodology more incisive, and they might decide to discard 
their present procedures. 

0 Inability to identify skills assessed in course. An objection we are likely to hear is that the skills and 
expressive modalities identified here are not relevant to a certain professor’s courses, that these things just 
aren’t what he or she teaches. Given the breadth of behavior and intellectual operations covered by the present 
model, such an assertion is dubious. Perhaps some instructors will need to consult with colleagues to link the 
appropriate labels with their exams, projects, and performances. Faculty workshops on assessment might also 
help resolve such problems of modality and skill identification. In the improbable event that someone’s target 
teaching/learning objectives actually do fall outside this comprehensive model, the modality/competency 
matrix does contain “Other” categories to accommodate variations. 



Conclusion 

The Course Profile approach to assessment is a universal method of gauging student achievement in two respects. 
First, it uses descriptive and measurement dimensions that cover virtually the full range of performance possibilities 
for human beings. Any competencies and intelligences not included in the standard categories can be specified and 
defined for particular disciplines under the “Other” rubric. Second, the approach requires only that professors express 
their normal methods of assessing students, and the results of those trials, in terms of identifiable competencies and 
intelligences. Thus, it is relatively painless, yet informative, to recast everyday measurement of student achievement 
within a framework of competencies/intelligences assessment. And since the instruments used are the professor’s 
usual tests and classroom-evaluation procedures, implications for changing teaching methods to enhance develop- 
ment of particular competencies and talents should be easy to see and act on. 
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Using Cross-Functional Teams to 
Develop an Electronic Portfolio 
to Assess Technical and 
General Education Outcomes 



Sherry A. Schwartz 
Mary L. Weisensel 
Peggy D. Kennedy 
John King 



Student Portfolio Assessment 

As a part of building a comprehensive academic assessment program at St. Paul Technical College, many forms of 
direct and indirect measures of student learning are being utilized. The overarching purpose of the assessment 
program is to continuously improve teaching and learning. Programs utilize many forms of traditional assessment 
measures, including written tests such as true-false exams, multiple-choice tests, and essay exams. Faculty are 
encouraged to utilize “non-test" measures, including case studies, projects, product simulations, and internship 
checklists. A recent innovation in measuring student learning is occurring through the use of portfolio assessment 
to demonstrate attainment of global and technical skills. 

According to Larson et al. [2000], student portfolios are particularly useful in and appropriate for the assessment of 
student learning in the technical college environment. Portfolio development is an alternative form of assessment that 
includes the assessment of active learning and performance ratherthan a focus on memorized facts. This is applicable 
to business and industry as well by building a connection between classroom learning and the world beyond school. 
The successful achievement of outcomes, however, depends on the purposes, practices, and protocols that guide the 
implementation of portfolio assessment. 

Portfolios have been used as a way for architects, engineers, and other professionals to illustrate their technical and 
global skill attainment and apply knowledge to practice. With colleges’ increasing emphasis on performance 
standards, student-centered classrooms, and accountability, the portfolio has become a collection of work samples. 
As an assessment tool, the portfolio should reflect the breath of study of the curriculum and the quality of work that 
students are expected to produce [Borthwick, 1995]. 



Developing Student Portfolio Protocols Using Crosp-Functional Teams 

Various forms of traditional portfolios have been used by St. Paul Technical College faculty in programs such as 
graphic design, human resources, and computer careers. Generally, the traditional portfolio has consisted of a resume 
and examples of student assignments. Due to the potential value of using student portfolios to assess technical and 
global outcomes, St. Paul Technical College explored ways to enhance the portfolio process, expand the use of 
portfolios across programs, and utilize innovative approaches to engage students in learning and assessment 
processes. 
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It was recognized that a cross-functional team would provide a mechanism for experts from across the college to come 
together and share valuable expertise. A cross-functional team consisting of technical and general education faculty, 
an academic dean, an instructional technologist, a curriculum specialist, and the assessment coordinator was formed 
to examine ways to enhance student portfolios and the assessment of program outcomes. The project leader helped 
facilitate the team’s purposes, roles, strategies, processes, feedback, and interfaces. The work plan was organized 
around the expertise of team members and the development of the portfolio components. 

The cross-functional team utilized assessment protocols recommended by L6pez 0996) to examine portfolio 
components and assessment measures to specify what the portfolio should include and how it should be assessed, 
by whom, and at what time intervals. The emphasis was on building a process for technical and general education 
faculty to collaborate on assessing technical and global outcomes through the use of assessment rubrics. A portfolio 
development course was designed to be team-taught by general and technical education faculty using multiple 
assessment measures of student learning. 

College-wide assessment measures for general education were established, and rubrics with specified criteria were 
developed. According to L6pez 0996), “Evaluators observe that the use of student portfolios to measure student 
learning is found effective by academic units that utilize them. They urge academic departments that use portfolios 
for assessment purposes to provide evaluation protocols in departmental assessment program documents as to how 
the portfolios are to be reviewed” [p. 13). 

An important priority of the cross-functional team was to offer students a traditional as well as an electronic option 
in the development of portfolios. Once the assignments and assessment protocols were established, the team worked 
extensively to provide a college-wide process for the electronic portfolio option. This required all members of the team 
to utilize their expertise to offer students with a wide variety of technology skills a means of documenting their 
achievements electronically. Technology procedures and resources were developed to provide students with options 
to use digital cameras, camcorders, scanners, and electronic file transfer. In addition, a process was established for 
students to document and demonstrate their technical and global skills through the use of video clips of oral 
presentations, recorded narratives, writing samples, and resumes. 



Benefits and issues of Implementing Student Portfolio Assignment and Assessment 
Protocols 

Student portfolios extend the basis of assessment beyond traditional forms, such as multiple-choice tests, to the 
alternative assessment of active learning based on clearly defined standards (Willis, 1996). Portfolio assessment 
provides instructors and students with multiple ways to diagnosis students’ strengths and weaknesses to help them 
improve their performance [Borthwick, 1995). When portfolio criteria are linked to program outcomes and provide 
students with clear expectations of technical and global skill requirements, they are an effective tool for helping 
students “see gaps in their learning, determine strategies thatsupport their learning, celebrate risk taking and inquiry, 
set goals for future experiences, and see change and development over time” [Porter and Cleland, 1995, p. 23). 

Portfolios also offer the benefit of engaging students in the learning process. Portfolio assessment is not instructor- 
directed, as are many other forms of conventional assessment processes. The portfolio process supports student- 
centered classrooms and a shared responsibility for assessment. The very nature of the portfolio assessment process 
involves students, instructors, and employers in establishing the criteria and content of the portfolio. It facilitates 
student involvement in and monitoring of learning and the documentation of progress and achievements over time. 

The portfolio process has the potential of engaging students as active learners throughout the educational process. 
A recognized value of portfolio assessment is that the process can accommodate the diverse learning styles of 
students and enable them to realize and experience success. 

Some of the issues regarding the difficulty of using portfolio assessment are related to reliability as a part of 
assessment protocols [Stecher et al. f 1996). Problems in scoring result from vague assignments and assessment 
specifications. Forthis reason St. Paul Technical College spent a significant amount of time developing recommended 
assignment and assessment protocols. Also of significance is the need for instructors to receive in-service on the 
portfolio assessment process and to be integral in all phases of the development process. It is important for students 
to be oriented to the portfolio process, components, assignments, and assessment. To be effective, assessment 
protocols must be established and utilized, including how assignments will be assessed, by whom, and at what time 
intervals. Clearly established assignment and assessment protocols are imperative. The technical and practical issues 
of portfolio assessment challenge educators to collaborate with other instructors on an ongoing basis and to involve 
students as active learners. 
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Reflections on the Student Portfolio Development Process 

Due to the complexity of building a college-wide portfolio process to document and assess technical and global 
outcomes and expand the use of portfolio options for students, a cross-functional team was utilized at St. Paul 
Technical College. The team was successful in achieving intended outcomes and developing traditional as well as 
electronic portfolio options for students. The cross-functional team achieved these goals by providing a mechanism 
for experts to come together and share valuable expertise. The work plan was facilitated by a team leader and 
organized around the expertise of team members and the development of the portfolio components. 

This approach will continue to be utilized at St. Paul Technical College for curricular and assessment initiatives due 
to the rich experience and well-received outcomes of the portfolio project. As colleges utilize technology in teaching 
and learning, an increasing number of cross-functional teams will be of value involving technology experts, faculty, 
instructional design specialists, and administrators to achieve results. 

According to O’Banion Cl 997), technology will continue to transform learning opportunities. Cross-functional teams 
serve as an important resource for technology imperatives across the college curriculum as experts come together 
to create innovative educational opportunities. Further, the use of cross-functional teams helps build esprit de corps 
across college personnel and new designs in teaching and learning. 
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Promise and Pitfalls 
of Electronic Portfolios: 
Lessons Learned from Experience 



Gloria M. Rogers 
Julia Williams 



Introduction 

A portfolio is a “purposeful collection of student work that exhibits the student’s efforts, progress, and/or 
achievements. The collection must include student participation in selecting contents, the criteria for selection, the 
criteria for judging merit, and evidence of student self-reflection.’’ 1 There is no one correct way to design a portfolio. 
The design should be driven by a clear understanding of the desired outcome from using portfolios and the knowledge 
and skills to be assessed. How the portfolio will be used will determine the design and focus of the portfolio 
development. Portfolios are not an end in themselves and must be developed with a clear vision of the desired 
outcome. 2 

Over the past few years, there has been an increase in the number of institutions using portfolios as a means to 
document and assess student work. 3 Many institutions have capitalized on the use of technology to create a more 
efficient portfolio process. Based on four years of experience developing and using portfolios at Rose-Hulman 
Institute of Technology, the decision to use portfolios in an electronic format is described. Based on the experience 
of implementation, both the promise and pitfalls of electronic portfolio use are discussed. 



Rose-Hulman Assessment Plan 

The Commission on the Assessment of Student Outcomes (CASO) was charged with developing an institutional 
assessment plan to provide information for quality improvement and satisfy the needs of external accrediting 
agencies. Student learning outcomes represent a significant portion of the assessment plan. Much of the 
development of the plan occurred in the summer of 1997. At that time, a subteam of CASO devoted particular attention 
to student outcomes assessment for the plan. The goal for student outcomes is to instill in our graduates the skills 
appropriate to their professions and lifelong learn ing. There are nine objectives that further specify these skills: ethics, 
teams, communication, global awareness, experiments, design, engineering practice, interpreting data, and contem- 
porary issues. Each of these skills has multiple, measurable, specific performance criteria that define the skill. 

Faculty researched various data collection methods that could be implemented. These methods included course 
grades, questionnaires and surveys, standardized tests, qualitative methods, and portfolios. Four main criteria 
developed for selecting the primary data collection method. The method should: 

1. Be rich, offering quality information about students in a breadth of outcome areas 

2. Produce valid results and reflect the uniqueness of the institution 

3. Be minimally intrusive on the time of students and faculty 

4. Serve students by engaging them in reflection on theirown education, and help them as they prepare for their 
careers or further education 
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Based on these criteria, the portfolio method was chosen. Because of Rose-Hulman’s computer-intensive environ- 
ment, it was determined that the portfolio system should be electronically based. 4 
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Promise 

The electronic portfolio CRosE-Portfolio) was designed and developed by Rose-Hulman faculty, staff, and students. 5 
The design was driven by how portfolios were to be used by both faculty and students. Having portfolios in an 
electronic format has four distinct advantages over using paper portfolios: 

1. Efficiency 

2. Asynchronous access 

3. Validation of process 

4. Adaptability 

0 Efficiency. Portfolio systems can be very cumbersome for both students and faculty to manage. By developing 
an electronic system to access, store, view, and rate student material, the amount of effort to manage the 
system is minimized. We were also able to design a system that integrates the student submission process, 
the rating process, the reporting process, and the curriculum mapping process Into one module. When 
students enter the system, they can easily access the list of student outcome objectives, see the rating rubrics, 
view online help, or submit questions and comments. 6 

0 Asynchronous access. The RosE-Portfolio system is web-based, and access is made through the local area 
network using the user’s network username and password. This allows both students and raters to access the 
system from anywhere at anytime they have access to the web. Students can make submissions to their 
portfolios from their homes, their residence hall rooms, or anywhere it is convenient for them at any time. This 
means that students can work on their portfolios at times when they are not so busy with their coursework. 
The system also allows raters to rate at their convenience against pre-established rubrics. Because teams of 
raters are involved, the system also provides for inter-rater reliability testing. This “calibration” of the rating 
process promotes the validity of the rating results. This aspect of the rating system will become increasingly 
important as we begin to involve alumni and business partners in the rating of portfolios. 

0 Validation of process. In designing the system requirements for the development of the software, it was 
found that the process of determining design requirements forced the Commission to think through the 
purpose and scope of the system in ways that actually promoted the efficiency and validity of the process. That 
is, by having to clearly articulate what the features of the portfolio would be, it was necessary to think through 
what we wanted students to do, why we wanted them to do it, and what we were going to do with the 
information. This was particularly true for the design of the rating module. The rating system was driven by 
the purpose of the ratings and howthe results would be used. By integrating all these decisions into the design 
of the portfolio itself, it was possible to continuously check our decisions against good assessment practices. 

0 Adaptability. One of the design constraints was for the system to be adaptable to other forms of 
documentation and assessment. As the institute-wide electronic portfolio system was developed, imple- 
mented, and improved, it was adapted for use in individual classes. Building on the existing system, some 
faculty modified the portfolio structure for both student teams and individual student submissions for learning 
outcomes that were specific to the course, in some cases moving to a paperless environment. Plans are also 
being made to prototype a system to be used for the faculty promotion, tenure, and retention process. The 
power of the electronic format and ease of adaptability serve to promote multiple uses of the system. 



Pitfalls 

In spite of the advantages of the electronic portfolio, there are also some pitfalls that need to be avoided. 

1. Too complex 

2. Inadequate design expertise/technology resources 

3. Technology overshadows assessment 

4. Short-term commitment 

0 Too complex. It is important to minimize the number of steps and the complexity of instructions in the use 
of an electronic portfolio. For the users of the system, both the design and the materials developed to support 
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the implementation of the system must be kept simple. The term user-friendly has to be a key aspect of 
developing materials to walk people through the process of submitting, reviewing, and rating portfolios. If the 
system is too complex to be easily understood, or if the materials developed are too complex, it will be difficult 
to get participation from both faculty and students. This is not to say that the electronic portfolio can’t be 
designed for multiple purposes and users. However, for the individual user, the complexity must be invisible. 

0 Lack of design expertise. Many campuses do not have expertise available to assist with the design of an 
efficient and effective electronic portfolio system. The responsibility for developing the system may be left with 
those who do not have the appropriate programming/technical expertise, or do not understand the importance 
of involving multiple constituents in the design process before developing the system. Without the appropriate 
expertise, it is unlikely that an electronic portfolio system can be designed to meet the needs of assessment. 

0 Technology overshadows assessment. For those who have both the resources and the desire to develop 
an electronic portfolio system, it is possible that the enthusiasm for the electronic side confuses the means 
and the ends. It is important to remember that the electronic nature of the portfolio is to enhance the portfolio 
process by making it more efficient and user-friendly. However, efficiency should not be at the expense of good 
practices in assessing student learning. Portfolios of any kind are not for novices. Whether the portfolio is 
electronic or paper-based, it is important to understand the purpose of the portfolio, what data are going to 
be collected, who is going to be responsible for submitting material to the portfolios, what rating criteria are 
going to be used, who is going to rate the material in the portfolio, how the rating process is going to be 
organized and managed, the use of the data collected, and how the students are going to get feedback. The 
answers to these questions help to drive the design of the system, and not the other way around. In any case, 
best practices for student assessment should be followed . 7 

0 Short-term commitment. When viewed as a short-term commitment, the development of an electronic 
portfolio system [as well as a paper-based system) is sure to fail. Because the commitment required to support 
and sustain a portfolio system is substantial, it is not an approach that should be entered into for the short- 
term. Resources need to be allocated for the design of the system. Faculty and students need to be educated 
about the use of the system. The system needs to be managed and nurtured. All of these processes take time 
to develop, implement, and improve to meet the needs of the institution. 



Lessons Learned 

The following lessons learned in the past four years can be applied to the task of developing an institution-wide plan 
for assessing student learning. 

0 Develop a common language. The importance of language as a means of conveying meaning cannot be 
overstated. This is especially true in the language of assessment in the current environment, where there are 
many debates about the role of assessment as it relates to accreditation requirements. Confusion over the 
meaning and use of the language of assessment often alienates the very people who need to be engaged in 
the development and use of assessment processes-the faculty. Because there are no commonly accepted 
standards of assessment terminology, it is important to develop a common vocabulary of assessment terms 
at the beginning of the process and to agree to use it consistently. 

0 Involve the key stakeholders. At the beginning of the process, identify who your key stakeholders are. This 
will vary from institution to institution. Everyone is important, but not everyone is equally important to 
determining what outcomes are appropriate for your institution. Don’t forget to involve students! 

0 Ask them, ask them, and ask them again. Faculty have a very long list of important things they have to 
do. Not all of them make it to the top of the list. It would be erroneous to assume that, just because you don’t 
get feedback from faculty, that they are not interested or do not want to have input. It is important to keep them 
informed of the progress that is being made and to solicit their input. 

0 Uncouple institutional assessment from faculty evaluation. Institutions have processes in place to 
evaluate faculty performance. The assessment of student outcomes for institutional effectiveness purposes 
should not be one of them. Faculty cooperation is critical to assessment, and if faculty feel that the process 
can be used against them, the likelihood of their participation is decreased. 
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0 All assessment questions are not equal. Use a common sense approach to planning. As the assessment 
plan begins to be developed, it is important to realize that you cannot do everything. There will be a lot of 
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interesting questions that you would like to have answered, but it is important to prioritize and begin with the 
most important questions. 

0 One size does not fit all. The pressure to develop assessment plans and the lack of expertise in institutional 
or program assessment can lead to the temptation to adopt another institution’s assessment methodology 
without seriously considering whether it is appropriate to the local environment. Much can be learned by 
looking at what others are doing to assess student learning outcomes, but it is rare that the plans and 
methodology of one institution can be adopted in total at another. 



Summary 

Deciding what methodology to use to assess student learning outcomes is an important aspect of assessment 
planning. The criteria for choosing an appropriate methodology should be carefully developed and tailored to the 
needs of the institution. Electronic portfolios can be both efficient and effective to assess student learning outcomes 
at a program or institutional level. When principles of best practice for assessing student learning are applied, the 
benefits for promoting institutionai effectiveness is greaiiy enhanced. inSiiLutions developing assessment plans and 
choosing assessment methodologies can strengthen their processes by reviewing the lessons learned by others. 



Notes 

1. Paulson, L.F., P. R Paulson, and C. Meyer. 1991. What makes a portfolio a portfolio? Educational Leadership 48(5): 60-63. 

2. Arter, J. A., et al. 1995. Portfolios for assessment and instruction. ERIC Digest No. ED388890. 

3. A national database of electronic portfolios can be found at <http://www.aahe.org/teaching/portfolio_db.htm>. 

4. Rose-Hulman has had a student-owned laptop program since 1995. 

5. For a complete discussion of the design of the Rose-Hulman electronic portfolio, see Rogers, G. M. and J. Williams. 
1 999. Building a better portfolio. ASEE PRISM , January: pp. 30-32. Rogers, G. M. and T. Chow. Electronic portfolios 
and the assessment of student learning. Assessment Update . 

6. On-line demo of the RosE-Portfolio can be seen at <http://www.rose-hulman.edu/ira/reps>. 

7. American Association of Higher Education. 9 Principles of best practice for assessing student learning 
<http://www.aahe.org/assessment/principl.htm>. 



Gloria M. Rogers is Vice President of Institutional Research, Planning, and Assessment at Rose-Hulman Institute of 
Technology in Terre Haute, Indiana. 

Julia Williams is Associate Professor of English and Coordinator of Technical Communications at Rose-Hulman Institute 
of Technology in Terre Haute, Indiana. 




228 



Chapter 10. Assessment of Student Academic Achievement: Tools for Assessment/ 229 



A Top Ten List of Assessment Tools for 
Academic Courses and Programs 



In May 1 999 the North Central Association of Colleges and Schools (NCA) visited the Air Force Academy. In the verbal 
out-brief to the Academy’s Superintendent, the accreditation team commented that the Academy’s assessment effort 
was “marvelous.” By employing multiple assessment techniques, the Academy is able to identify patterns of 
convergence in data as a basis for programmatic changes. The purpose of this session is to highlight the variety of 
assessment tools in use to accomplish ongoing assessment of the Academy’s academic program at both the course 
and departmental levels. The presented tools have broad applicability in a variety of settings. 

We were able to identify several unique program assessment tools by reviewing the Academy’s Assessment Catalog. 
The catalog provides an at-a-glance summary of assessment efforts related to cadet academic achievement, 
performance of Academy graduates in the Air Force, and performance of the departments and agencies in their 
educational mission. The catalog was first created in 1997 as a product of an administrative initiative to document 
assessment efforts already in place. The USAFA Assessment Catalog serves several purposes. It allows for cross-flow 
of assessment ideas between and among departments and agencies; identifies internal and external sources of 
assessment data; allows for easy identification of qualitative and quantitative assessment methods; categorizes 
assessment instrument types; tracks the currency and frequency of use of assessment methods; identifies decisions 
based on assessment data; provides judgments about the utility of the assessment methods (low, moderate, high); 
and identifies knowledgeable points of contact within departments and agencies. We carefully examined the entries 
to select tools that could be easily modified by other institutions and programs. 

According to Palomba and Banta, “a key to assessment success is involving faculty in the process” (p. 1 0). The views 
and voices of faculty members need to be considered when framing questions and identifying topics of inquiry. In 
many instances, faculty are responsible for designing and implementing an assessment plan as well as reporting 
results and making recommendations for programmatic changes. Students are also key players in the implementation 
of a successful assessment plan. The tools presented grew from a bottom-up implementation at the Academy. Each 
academic department designs and implements tool to meet its specific needs. Faculty involvement is high, and 
students are key players in many of the assessments. 

Some of the more unique tools to be discussed include: 

o Structured focus groups 
o Archived course records 
o Student management teams 
o Surveys and interviews of graduates 
o Study time and workload surveys 
o Grade histories 
o Anchored exams 



Marie A. Revak 
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The overriding purpose of assessment is to understand how educational programs are working and to determine 
whetherthey are contributing to student growth and development Hence, the ultimate emphasis of assessment 
is on programs rather than on individual students. 



- Catherine A. Palomba and Trudy W. Banta, Assessment Essentials 
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o Grading exercises 
o Visiting faculty feedback 
o Curriculum benchmarking. 



Every assessment tool comes with own strengths, weaknesses, strategies, guidelines, data analysis techniques, and 
uses. The assessment tools listed above have broad application potential in a variety of settings. Below, we’ve 
addressed these issues for five of our favorite program assessment tools. During our session, we'll share some of the 
actual assessment instruments, discuss logistical procedures, and share example reports. The remaining five tools 
will be discussed in a similar manner during our presentation. 



□ Structured Focus Groups 

Purpose: To involve students in the course in program assessment through ah interactive 
medium. 

Strengths: Feedback is deeper and richer thah what can be obtained using traditional surveys and 
can provide insights that might not occur without interaction between students. 
Students feel that they have a voice. Method is more efficient than individual interviews. 

Weaknesses: Requires a time commitment on the part of the students and the facilitator. Requires 
mature, responsible student participation and skill on the part of the facilitator. 
Requires substantial planning. Shy or minority students may be reluctant to partici- 
pate. Students may suggest unreasonable changes to the course or program. 

Strategies and guidelines: Select a representative sample of 15-20 students. Follow the protocol developed at the 
Air Force Academy to include audiotaping of the focus group session. 



Data analysis techniques: Qualitative analysis of the focus group transcript to include the identification of trends. 

Quantitative analysis of student satisfaction data and counting of trend data. Can be 
correlated with student ratings data. 



□ Archived Course Records 



Purpose: To capture a snapshot the entire course and document changes to the course over 
several semesters. 



Strengths: Helps maintain consistency in course content and delivery in the context of rapidly 
changing organizations. May be used by faculty to update their academic portfolios. 
Provides corporate knowledge. Easily accomplished immediately after the closeout of 
the semester. 



Weaknesses: May challenge the perception of academic freedom. Can be time-consuming if not 
accomplished immediately. Sufficient document storage space required. 

Strategies and guidelines: Collect and assemble all course-related documents and products to include syllabus, 
handouts, formative and summative assessments, student evaluations, and other 
feedback. Also included are course administrative documents such as a list of 
instructors, the title and edition of the textbook, a summary of curricular changes, and 
a historical course grade summary. 



Data analysis techniques: Qualitative analysis of changes, themes, and trends in both curriculum and evaluation. 



□ Student Management Teams 

Purpose : To involve students in the course in program assessment while the course is ongoing. 

Strengths: Feedback is continual and allows for changes during the current offering of the course. 

Students have a voice, and the instructor has a mechanism for requesting and receiving 
feedback on a regu lar basis. Allows for the collection of richer, deeper assessment data. 
Instructors may utilize the Student Management Team to address specific concerns. 

Weaknesses: Requires a time commitment on the part of the students and faculty member. Requires 
mature, responsible student participation. May require a reward mechanism for 
students to ensure quality feedback. Students may suggest unreasonable changes to 
the course or program. 
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Strategies and guidelines: May use volunteers, or instructor may appoint 3-4 members to the Student Manage- 
ment Team.Teams meet once per week with the instructor and may schedule additional 
meetings with or without the instructor. 

Data analysis techniques: Qualitative summary of the accomplishments of the team and possible correlation with 
student satisfaction ratings and or outcomes. 



□ Surveys and Interviews of Graduates 



Purpose: To assess courses and programs from the perspective of students after they have 
entered the workforce or graduate school. 

Strengths: Provides a different perspective than currently enrolled students. Serves as a validity 
check for student survey data. Fosters alumni relations. 



Weaknesses: Low participation rates due to low motivation and hard-to-find contact information. 
Need to determine appropriate wait time and intervals for survey repetition. 

Strategies and guidelines: Need substantial administrative support to develop, administer, collect, and analyze 
survey data. Survey must be as short as possible while still tapping the most important 
information. Multiple methods of response may be necessary. Plan for dissemination of 
results to survey participants. 



Data analysis techniques: 



Both quantitative and qualitative depending on survey questions. 



□ Anchored Exams 



Purpose: Common final exam items allow comparison of student performance on key topic 
outcomes over the course of several semesters. 

Strengths: Helps maintain consistency in core course content in the context of changes in the 
curriculum and teaching personnel. Based on final exam items, benchmarking is easily 
accomplished immediately after the closeout of the semester. Most applicable to 
courses with a relatively stable content. 

Weaknesses: May challenge the perception of academic freedom. Could inhibit timely changes to 
rapidly changing curriculum. May lead to the testing of less-important skills and 
concepts. May not apply to courses undergoing major curricular reform. 

Strategies and guidelines: Assemble valid and reliable test items for inclusion in final exams over the course of 
several semesters. Conduct test item analysis on each item, track student performance, 
and match items to established course criteria. 



Data analysis techniques: Statistical comparisons of performance on test items. Test item analysis data can also 
be examined for changes in item difficulty and selectivity. Map items to course criteria 
to determine whether course goals and objectives are being met. 



Essential to any comprehensive assessment program is the triangulation of data and its subsequent use in curricular 
decisions. By employing multiple assessment techniques and collecting both quantitative and qualitative data, 
assessors are able to identify patterns of convergence in data as a basis for programmatic changes. The assessment 
methods presented during this session actively involve faculty and students and can be tailored to specific course 
and program goals. The tools can be used to answer locally generated questions, meet program assessment and 
accreditation requirements, and aid the curricular decision making process. 
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Test Anxiety and Meeting the Mission: 
Recognizing Necessary Institutional 
Resources and Overcoming the 
Resistance of Using Tests to 
Assess Student Learning Outcomes 



Introduction 

The College of Saint Benedict (CSB) and Saint John’s University (SJU) are two liberal arts colleges located four 
miles apart in Central Minnesota. Saint Benedict’s is a college for women and Saint John’s is a college for men. 

The students of these two colleges share in one common education, as well as coeducational social, cultural and 
spiritual programs. ... CSB/SJU have a common curriculum, identical degree requirements and a single 
academic calendar. All academic departments are joint, and classes are offered throughout the day on both 
campuses. The academic program is coordinated by the Provost for Academic Affairs, who is assisted by 
undergraduate academic deans on each campus (CSB/SJU Academic Catalog, p. 4). 

Our NCA site visit for comprehensive evaluation occurred in November 1998 on both campuses. Although we were 
re-accredited, areas needing further attention included general education assessment and assessment in academic 
departments. A required progress report was submitted in December 2000. This paper describes (1) our transforma- 
tional strategies, which overcame our institutional roadblocks to productive assessment, and (2) our effective usage 
of standardized and locally developed assessment tests resulting in curricular revisions. 

Transformation of Campus Culture 

□ The Past 

In the recent past, our faculty resisted the challenge to assess student learning. Some considered assessment and 
review an encroachment on their academic freedom. Curricular changes within departments were often based on 
hunches by faculty based on their own educational experiences and traditions, rather than assessment and 
program review outcomes. Central to our recent attitudinal transformation is a shift in our and the faculty’s 
understanding about assessment. We have had conversations at various levels about what assessment of student 
learning is, what assessment data are, and howwe can use assessment to improve the curriculum. We have moved 
from a focus on faculty teaching to a focus on student learning. (Huba and Freed, 2000; L6pez, 1999; Knox and 
Knuesel, 2000). In Table 1 we list the strategies we developed to overcome our specific roadblocks. 

Although these strategies moved our campuses to a state of readiness for active assessment work, we still 
struggled with our assessment efforts. 
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Table 1. Our Inventory of Needed Assessment Resources and Strategies 



Inventory 


Roadblocks 


Strategies 


Faculty 

Involvement 


Many faculty dubious about 
worth of assessment. Perva- 
sive faculty attitude of “You 
can’t make me assess.” 


Created common understanding of (1) what is 
assessment of student learning, ( 2 ) what are 
assessment data, and (3) how we can use our 
findings to improve the curriculum. 


Assessment 

Committee 


Pitiful and anemic. Faculty 
with no expertise or interest. 


We created a new and enhanced assessment 
committee. Senior faculty with appropriate exper- 
tise were appointed to the committee. Committee 
members were sent to the NCA Annual Meeting to 
further develop expertise. Provost is now an active 
member of the assessment committee. 


CAO 


Inadequate provost support 


We hired a new provost who is knowledgeable, 
invested and experienced in assessment and pro- 
gram review practices. In his inaugural address, 
he set his NCA agenda for the next two years. 


Money 


Inadequate funding. 


Ongoing funding established by institution-wide 
commitment and reallocation of resources. 


Personnel 


No one person was respon- 
sible. 


The academic deans became highly involved 
and initiated the renovation of our assessment 
and program review processes. The deans and 
provost created a part-time facilitator of program 
review and assessment, who is a member of the 
assessment committee. 


Program 

Review 


No regular program review 
process in place. 


Provost worked with standing faculty committee 
to finalize program review policy. Deans and com- 
mittee established timetable for program review. 


Strategic Plan 


Assessment and program 
review absent from strategic 
plan. 


Strategic plan rewritten to include assessment 
and program review initiative. 


Communications 


No regular, deliberate com- 
munications. 


Chair of Assessment Committee regularly re- 
ports at faculty meetings and presidents’ cabinet 
meetings. Assessment Committee minutes are 
posted in an intranet folder available to all com- 
munity members. 



□ The Mission 




To our surprise, we discovered that many of our faculty leaders of assessment did not understand the fundamental 
connections between the institutional mission statement the departmental/program mission statement the 
departmental assessment plan, and the process of assessing student learning. Without this basic understanding 
that one leads to the other, our institutions continued to suffer assessment paralysis. The task of assessment 
seemed too large and ambiguous because the touch-point of assessment was missing. We should be testing 
whether we are meeting our mission statement about student learning. When this fundamental concept was finally 
realized, assessment became easier because improved strategies became obvious. Suddenly, assessment 
outcomes became real; curricula were improved within departments and programs; and the ownership of success 
and pride of meeting the challenge resulted in a boost of institutional morale. 
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□ Choosing Tests 

Our departments and programs also floundered with incorporating assessment activities into their routines 
because they believed they had to independently develop their own assessment instruments from scratch. Using 
the combination of standardized and locally developed tests (1) enabled departments and programs to move 
forward with assessment strategies, and (2) allowed comparisons with national reference groups without losing 
the ability to study the uniqueness of our departments/programs. In Table 2 we list the standardized and locally 
developed tests we have used to date. In Table 3 we explain the source of the tests. Assessment results and 
curricular modifications as a result of assessment testing were reported to the institutional assessment committee, 
whose wide membership includes faculty with expertise in test measurement, the academic provost, one academic 
dean, the facilitator of assessment and program review, student representatives, and a student development 
representative.This committee wrote our NCA progress report. 



Table 2. Departmental Usage of Standardized and Locally Developed Tests 





Standardized Tests 


Locally Developed 
Tests 


Assessment Tools 
Inventory 


MFT 


ACS 


CAP 


CL 

< 


Praxis II 


CPA Licencure 


PSAS 


AASSWB 


NCLEX 


GRE (subtest) 


TUCE 


Writing Tests 


Portfolio 


Adjudication 


Language Placement 


Juries 


Disciplinary Content 


Departments and Programs 


Accounting 












S 
























Art 


























S 










Asian Studies 




































Biology 


S 
































S 


Chemistry 


S 


s 










A 






















Communications 














S, A 






















Computer Science 


s 


































Economics 














S, A, 

F 








s 














Education 








s 


s 














s 


s 










English 
























s 


s 










Environmental Studies 














A 






















Gender and Women’s Studies 














< 

CO 






















History 
























s 












Management 


s 












< 

CO 












s 










Mathematics 


s 


































MCL 






























s 






Military Science 




































Music 


s 






























s 




Nursing 






S 




E 




S, F, 
C, P, 
S 






















Nutrition 




































Peace Studies 




































Philosophy 





































0 



S=students; A=alumnae/alumni; F=faculty; C=clinical site; P=patient; E=employer 
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Table 2. (continued) 


Standardized Tests 


Locally Developed 
Tests 


MFT 


ACS 


CAP 


& 

< 


Praxis II 


CPA Licencure 


PSAS 


AASSWB 


NCLEX 


GRE (subtest) 


TUCE 


Writing Tests 


Portfolio 


Adjudication 


Language Placement 


Juries 


Disciplinary Content 


Physics 




















s 
















Political Science 


s 






















s 












Psychology 


s 


































Sociology 




































Social Work 
















s 




















Theater 














A, S 














s 








Theology 


































s 


Physical Education 




































Liberal Studies 




































Academic Skills 




































Honors 




































Natural Science 




































Social Science 




































Humanities 





































S=students; A^alumnae/alumni; F=faculty; C=clinical site; P=patient; E^employer 



Table 3. Assessment and Program Review Tests 



Standardized Tests 


AASSWB 


American Association of State Social Work Boards 


s 


ACS 


American Chemical Society 


s 


AP 


Academic Profile 


s 


CAP 


California Academic Press-Critical Thinking 


s 


GRE 


Graduate Record Exam 


s 


MFT 


Major Field Test 


s 


NCLEX 


National Council Licensure Examination-Nursing 


s 


Praxis II 


Teacher Licensure 


s 


PSAS 


Program Self Assessment Survey 


s, F, A 


TUCE 


Test of Understanding of College Economics 


S 


Locally Developed Tests 


Adjudication 


Theater only 


S, F 


CAT 


Classroom assessment techniques 


S 


Juries 


Music only 


S 


Language Placement 


Foreign language only 


S 


Portfolio 


Systematic scoring rubric-program-specific 


S 


Writing Tests 


Faculty scored with systematic rubric 


S 
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□ Curricular Revision 

We cite two of many possible examples of immediate curricular improvements as a result of using assessment tests. 

Using two locally developed tests/assessment methods, the English department discovered their majors were 
weak in grammar skills. As a result, a new two-credit grammar class required of all majors is being developed to 
remedy this inadequacy. While many programs would consider this an old-fashioned value, our English 
department regards knowledge of sentence and paragraph construction critical to writing well and essential to 
their departmental mission of student learning. Another of their concerns was how multiple professors would 
evaluate students’ portfolios using a standard rubric. Their remedy was to schedule and organize their own 
workshops attended by all English department members to develop their common evaluation methods. These 
workshops have further cemented their sense of common purpose and solidarity. 

In theater, an outside practitioner was brought in to adjudicate our four productions. Portfolios of writing-i.e., 
samples from criticism and history courses, resumes, photo documentations of scenic and costume classes, etc.- 
were analyzed. Alumnae and alumni surveys were conducted, and focus group studies of our current students were 
conducted by one of our assessment experts on campus. We discovered that our technical classes and technical 
experiences did not have enough hands-on opportunities for our students. Sound and lighting was not addressed 
strongly in our theater curriculum. The theater faculty have decided that the second technical class will have sound 
as part of the curriculum and will require the students to participate in the light hang for both touring companies 
and our production. An unexpected outcome came about because we used the National Association of Schools 
of Theater [NAST) accrediting standards in our assessment and program review efforts. The department has asked 
the administration to investigate whether we would like to apply for this theater accreditation. 



Conclusion 

It became evident on our campuses that personnel and physical resources needed to be identified to assure 
successful assessment and program review strategies. Recognizing our mission statement and its immediate role in 
assessment and program review strategies was critical in creating a common understanding of meaningful 
assessment activities. The realization that we could and probably should use multiple methods to test student learning 
moved our campuses from a culture of fear and resistance to a culture of action and accomplishment. 
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The Electronic Portfolio as an 
Alternative Assessment Model for 
Graduate Education 



Barbara L. Nicholson 

Why might a student in a presen/ice educational administration program find the development of a professional 
portfolio useful? For some, the primary rationale would be compliance with institutional requirements for obtaining 
their master’s degrees. There are, however, other reasons to consider the beginning of a professional portfolio at the 
preservice stage, some functional, others more philosophical. This paper provides an introduction to the practical and 
theoretical elements involved in the development of the Marshall Preservice Principal Portfolio [MP3). Before turning 
to the MP3 itself, however, the benefits of beginning the construction of a portfolio during the preservice phase of 
an administrative career merit some attention. It is these benefits-an ability for self-assessment, the value of 
establishing a reflective practice, and the potential to transform pedagogy-thatform the conceptual foundation on 
which the Marshall Preservice Principal Portfolio rests. 



Conceptual Issues 

□ Self-assessment 

Recent years have seen a number of challenges to the orthodoxy implicit in assessment rhetoric from kindergarten 
to postgraduate classrooms, their universal intent being the improvement of public education. In higher education, 
this discourse has been energized by a variety of progressive approaches, among them an increasing acceptance 
of qualitative measures that are perhaps less authoritative but more illuminating than the quantitative measures 
that have dominated the field literally for decades. Among these approaches is portfolio assessment. While still 
primarily used in conjunction with more conventional evaluation methods [e.g., comprehensive examinations, 
certification tests, and performance appraisal instruments), the portfolio has been gaining the support of both 
students and faculty across disciplines, and it has had a significant impact on teacher education programs 
[Darling-Hammond et al., 1 995; Holmes Group, 1 986; Lyons, 1 998; Moss, 1 997; National Board, 1 989; Schon, 1 987; 
Schulman, 1998; Tierney, 1992). 

The implementation of the portfolio as either an alternative or correlative assessment tool in educational leadership 
programs, however, has been less ubiquitous. This has been a puzzling phenomenon to some who work with 
preservice administrators, particularly in light of the body of research concerning the centrality of the principal’s 
performance to school effectiveness. Despite what we know about the principal’s integral role in various 
effectiveness measures, the majority of preservice administration programs have been characterized by the 
dubious premise that mastery of a prescribed knowledge base adequately prepares principals for the instrumental 
problems they will encounter in leadership practice. This approach has not only divorced propositional knowledge 
[ knowing what) from procedural knowledge (knowing how), but has also privileged the former to the virtual 
exclusion of the latter. The challenge for both those who teach and those enrolled in educational leadership 
programs is to reconcile these competing philosophies with an epistemological model that recognizes the value 
of both. The development of a preservice portfolio can help not only to effect that reconciliation, but also to provide 
a foundation for the continued growth and development of the practicing administrator. 

□ Establishing a Reflective Practice 



0 
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It is ironic that a discipline that at the beginning of the twentieth century was exhorting its students to adopt what 
Murphy (1 995) calls “the captain of commerce role” (63) is now witness to some of the better known management 
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theorists’ emphasizing business models predicated on educational premises. One of the more recognizable is Peter 
Druckerwho, in 1989, was describing management as a liberal art: “Management is... ‘liberal’ because it deals with 
the fundamentals of knowledge, self-knowledge, wisdom, and leadership; ‘art’ because it is practice and 
application” [231) 

Unlike propositional knowledge, self-knowledge is not easily assessed in conventional ways. Sergiovanni [1995)argues 
thatindividualsaspiring to be effective principals need to knowwhatthey value, whatthey’ re committed to. It isthisquality 
to which most refer when they employ the term vision. Values, commitment, and vision, however, do not inhere in the 
content of a knowledge base, regardless of its breadth or depth. They grow in the spaces we carve out for ourselves 
to spend in contemplation when we begin to sense an incongruity between what we believe and what we are doing, 
i.e., in the friction of theory misaligned with practice. In the well-documented fragmentation of the principal’s day, 
however, there’s little time for this kind of reflection, what Schon [1987) calls “reflection-in-action” [22). 

Recognizing that the cultivation of a habit requires repetition, the MP3 is predicated on the premise that ongoing 
reflection, from the beginning of the student’s program to the end, lays the groundwork for the extension of that 
behavior to professional practice. Underlying this premise is a constructivist foundation that identifies reflection 
as essential to the student’s ability to continue to evolve as a self-directing, inquiring learner, and that recognizes 
lifelong learning as not only valuable, but as imperative in a world where information with knowledge potential 
grows exponentially on a daily basis. 

Neither information nor knowledge stands still. As information has accumulated, scholars have added to their 
knowledge bases in every discipline by making reasoned judgments about what constituted a valuable addition 
and what didn’t. That task grows ever more difficult, however, as distinguishing what information is useful from 
what isn’t, and determining how we’ve come to know that, gets increasingly complicated. That process is 
understood as constructivism, and reflection is its essential quality. 

One becomes a lifelong learner through understanding and awareness of the processes she/he engages in when 
she/he “comes to know.” The identification of those processes is the product of thoughtful deliberation, i.e., 
reflection, which then “enhances [one’s] ability to [continue to] learn and make sense of new information” 
(Lambert et al., 1 995: 1 8). Schon (1 987) argues that reflection is central to growth and development, that it is the 
ability to reflect on learning that allows students to construct the theoretical frameworks that will guide their 
practice. This awareness of the processes through which they’ve absorbed what they know imbues learning with 
a new character, making it dynamic, individually meaningful, and, most important, autonomous. 

It is autonomy that separates constructivism as an epistemological construct from previous conceptions of how 
students learn. Previous epistemologies have for the most part construed students as beneficiaries in a sort of 
philanthropic relationship with their professors. Students came to “receive” the knowledge their professors “gave” 
to them, rendering them essentially passive participants in the learning process. As opposed to knowing subjects, 
they were cast as uninformed objects. Constructivist theory, however, consistent with Dewey’s (1916) character- 
ization of the student as a self-directed being, shifts the responsibility for learning from professor to student. 

Embedded in that responsibility is the need for self-assessment. The more conventional forms of assessment (e.g., 
course, comprehensive, or licensing examinations), as Lambert et al. (1995) point out, “reinforce the notion that 
knowledge exists outside the student, and that the teacher’s role is to transmit, and test the ... acquisition of 
knowledge” (24). Portfolio assessment, however, gives the student a pivotal role. It is a process characterized by 
a shift from what Chittenden and Gardner (1991) call a testing culture to an assessment culture; one in which 
assessment is continual and integrated, as opposed to transient and fragmentary. This shift represents a significant 
change in our understanding of assessment, one that is best facilitated not by a culminating examination, but by 
students’ critical assessment of and reflection on the experiences they have as they prepare to become principals. 

□ Transforming Pedagogy 

The potential of the portfolio to resolve competing claims is not limited to epistemology, although its epistemo- 
logical dimensions may be of most benefit to the individual student. There are aspects to the use of the portfolio 
as a programmatic assessment tool that make it valuable to institutions as well. Critical pedagogical issues for 
administrative preparation programs are frequently obscured, deliberately or not, by debates concerning academic 
questions of dubious importance. Among those critical issues is the profound sense of dissatisfaction with 
contemporary educational administration programs, which are characterized as being largely irrelevant and out 
of touch with practical concerns. 
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In addressing the historical underpinnings of educational administration, Murphy [1 995) believes we allowed that 
rift to develop out of deference to the persuasiveness of the scientific model and in response to the abuses of what 
he characterizes as the “prescriptive era” in the evolution of the knowledge base. The result, he argues, is this: 

Knowledge was something that was created at the university and applied in the field. It was a nonrecursive 
relationship. As a result of this academic self-conceit, a distinct breach developed between the university 
and field dimensions of the profession, one incorrectly and arrogantly labeled the theory-practice gap. The 
officially sanctioned knowledge base became increasingly less useful to practitioners. Worse yet, the 
processes and procedures employed to transmit this knowledge were often diametrically opposed to 
conditions that characterize [sic] the workplace in which school administrators found themselves. [69] 

Murphy’s analysis is correct, of course, and that’s a shame. It is a problematic situation for both the university 
professor, who must necessarily concern herself with the constraints of institutional support and the retention/ 
promotion/tenure process, and the student of administration who, having his graduate education reduced to an 
exercise in instrumentalism because certification requirements tend to force administrative preparation programs 
into a one-size-fits-all shape, resigns himself to enduring hours of pointless study in the interest of getting his 
credential. While there are always exceptions, such a circumstance would appear to be the rule. Were it not, we 
wouldn’t continue to thrash about trying to find ways to ameliorate it. 

While a broad discussion of strategies for eliminating institutional constraints and the instrumental quality of 
certification requirements is beyond the scope and intent of this paper, it is possible to examine the portfolio as 
one small reform that has the potential to at least mitigate their impact on the relationship between universities 
and schools as well as between faculties and students. If it is true that reforms generally falter on their failure to 
conform to the self-interest of those involved, the portfolio has an excellent chance to succeed. 

If it is the gap between university values and elementary and secondary school needs that keeps the two at cross- 
purposes, a mutual focus on the world of practice can serve to redefine the gap by making it common ground. If 
we reconceive of pedagogy not as the art or science of teaching but as lying at the intersection of teaching and 
learning, a discursive space wherein each informs the other, the potential of the portfolio to reform pedagogy 
begins to take shape. Supplemented by corresponding programmatic transformations (e.g., a redistribution of field 
experiences throughout the program, accompanied by reflective entries developed in collaboration with field 
mentors and faculty], the portfolio can create multiple opportunities for a discursive relationship between 
universities and schools and between faculty and students that is pedagogically rewarding; one in which 
knowledge of what is useful in the world of practice can be jointly constructed. Students can play an integral in 
that transformation. 



Practical Matters 

The choice to use an electronic format for portfolio construction was made for its congruence with the project's 
conceptual principles as well as for practical reasons. As emerging technologies expand the dimensions of the 
classroom, demands that education professionals be familiar with not only their potential but their application 
increase as well. Because what is required of an individual to be technologically literate is something of a moving 
target, creating the portfolio in a constantly changing digital environment constitutes the kind of ongoing learning 
process encouraged by constructivist pedagogy. 

From a pragmatic perspective, there are two additional benefits which accrue to the student who assembles a 
preservice portfolio. First, the electronic format allows for portability and ease of handling [for both students and 
university files); expedites editing; provides options that are either unavailable or awkwardly achieved in conventional 
portfolios [e.g., qualitative research can capitalize on video streaming, audio recording, and scanning documents as 
opposed to submitting reams of paper and video and cassette tapes); includes a broad range of statistical packages 
for quantitative research; simplifies backup; and represents a kind of compact credential for potential employers as 
it can be either copied to a zip disk or pressed to a compact disk. 

Second, the number of states that require portfolio evaluation for practicing administrators is growing. Given the 
number of accrediting agencies that now recommend the process, an increased interest on the part of state agencies 
that certify or license school admin istrators can be detected as well. Among the benefits cited to Brown and Irby [1 997) 
by practicing administrators who had constructed portfolios were opportunities to reflect on their experiences by 
documenting their strengths, collecting evidence of their accomplishments, identifying their weaknesses, setting 
goals for improvement, and planning for their own professional development based on those goals [53-54). 
Leithwoodand Montgomery [1986) suggest that the ability to reflect on practice is central to an effective principalship, 
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particularly as it relates to problem-solving. Development of a preservice portfolio provides a foundation on which 
this kind of reflective practice can rest. 

Maclsaac (1991) makes a useful distinction between the portfolio and the folio, both of which are featured in the Marshall 
Preservice Principal Portfolio. The portfolio contains only those entries chosen by the studentto demonstrate proficiency 
or comprehension of a specific set of skills or body of knowledge. The folios are less discriminatory, including the 
accumulated body of work for a course, project or program. The student’s MP3 disk will include four folios: 

o The course folio (with a folder for each course in the program, each including the assignments the professor- 
of-record identified as folio artifacts): 

o The field experience folio (with a folder for each of the field-based experiences required in conjunction with 
each course, plus the student’s reflective evaluation of each experience); 

o The reflective folio (six essays corresponding to the ISLLC standards/principles): and 

o The ancillary folio (evidence of the student’s learning in other environments, e.g., school improvement councils, 
school-community meetings, conferences, and workshops). 

The disk will also include the student’s portfolio, i.e., selected a rtifacts from the folios that provide evidence of the her/ 
his growth and development throughout the program. The final step, grounded in the work of Vygotsky (1978) and 
Bruner (1994), who argue that we come to know ourselves primarily through our interactions with others, is a public 
seminar in which MP3 students will present the completed portfolios. The setting is designed to engage students, 
faculty, peers, and mentors in a collaborative inquiry that ultimately invites all participants to revisit their own 
leadership philosophies: the goal is to internalize the explaining, elaborating, and interrogating of our respective 
understandings in a learning community as an ongoing element of professional practice. 

It is, of course, in the implementation of assessment reform that the larger challenges exist. The transition from an 
incremental program, wherein students move in a linear fashion through a prescribed sequence of courses, an 
internship, and a final examination, to one that is more holistic in nature, combining coursework with field-based 
experiences and relying on self-assessment, is not simple. 

For faculty, issues of epistemological appropriateness, theoretical underpinnings, and assessment validity are among 
the matters of concern. Possessing the skills necessary to work in an electronic environment may constitute a 
complication as well, and development of assessment rubrics is especially difficult. Students faced with the task of 
creating an instrument through which they themselves must choose how to demonstrate their capabilities will 
experience some uncertainty as well, and many may share the same technological anxieties as their professors. Full 
consideration of these and other questions must be handled in a more immediate context, i.e., faculty development 
sessions and student workshops or seminars. However, those involved in the Marshall Preservice Principal Portfolio 
project- students and faculty alike-have found the transition beneficial. 

The portfolio is at the center of an assessment model predicated on self-designed inquiry and realized through 
clarification and justification. At its center is the transformation of leadership from technical skill to theoretical act, 
undertaken and sustained by the students themselves. It is they who will enable that transformation, who will look 
critically at themselves and their work, and who will act on that critique to improve their practice. Such reflection, as 
Bolin (1988) notes, is not a given of the human condition. It should be. The goal of the portfolio is to create an 
opportunity for its internalization. 
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How It Works: 
Assessing Student Learning 
Across the Curriculum 



Cheryl Frank 
Joan Kuzma Costello 
David A. Shupe 



What Is the Liberal Studies/Professional Skills Program? 

The Liberal Studies/Professional Skills program [LS/PS) is a model used by Inver Hills Community College OHCC) to 
assess student learning across the curriculum. It is a way to measure what skills students master as they complete 
their coursework. LS/PS emphasizes both the value of traditional liberal arts and the professional capabilities needed 
to perform successfully in the twenty-first century. It helps students see the connection between the skills they learn 
in their courses and the applicability of these skills to their world. It focuses on what students have learned to do within 
the context of particular courses and provides students with a Skills Profile-as a complement to their transcript-that 
verifies what skills they have mastered. 

For more complete information on the LS/PS program, visit the LS/PS website at www.inverhills.mnscu.edu/lsps where 
you can see a sample Skills Profile , the complete Essential Skills grid ' rubrics for evaluating student performance, and 
other information pertinent to the program. 



What Are the Ten Essential Skills? 



The model begins with a list of ten Essential Skills that, combined with the knowledge that comes from a college 
education, provide students with the basis for lifelong learning, service, and employability in contemporary society. 
These skills are also characterized as: 



o Transferable: Once learned and practiced in one context, they can be adapted and applied in other contexts, 
o Professional: They are the basis of most professional work, 
o Core competencies: Once mastered, they become a part of who a person is. 

Each of the Essential Skills is useful in an academic context, a business context, and a community context. For instance, 
the skill of understanding another point of view applies in understanding persons studied in a text [academic context), 
customers [business context), and neighbors [community context). Each of the Essentials Skills can be developed: 
that is, it can be modeled, taught, learned, assessed, and measured by successful demonstration. 



The ten Essential Skills are: 




1 . 

2 . 

3 . 

4 . 



Appreciation skills: 
Collaboration skills: 
Conceptual skills: 
Implementation skills: 



Achieving a perspective on oneself and the world 
Working with people effectively 
Organizing ideas and materials 
Working and organizing effectively 
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5. Inquiry skills: 

6. Material skills: 

1. Presentation skills: 

8. Qualification skills: 

9. Quantification skills. 
1 0. Technology skills: 



Finding, assimilating, and evaluating new information 
Interacting with the physical world 
Communicating effectively and responsively 
Making distinctions and finding connections with words 
Making distinctions and finding connections with numbers 
Using technology 



In addition, each of the ten skills is further divided into five subcategories. 



1 . Appreciation skills: Achieving a perspective on oneself and the world 

o Others’ viewpoints 
o Breadth of vision 
o Attention to excellence 
o Commitment to principles 
o Personal accountability 



2 . Collaboration skills: Working with people effectively 

o Service to others 
o Teamwork 
o Group project 
o Negotiation 
o Mediation 



3 . Conceptual skills: Organizing ideas and materials 

o Classification structures 
o Theories/hypotheses 
o Models 
o Formal documents 
o Plans 



4. Implementation skills: Working and organizing effectively 

o Self-management 
o Development 
o Organizational management 
o Protocol 

o Adaptation 

5 . Inquiry skills: Finding, assimilating, and evaluating new information 

o Learning through observation 

o Learning from people 

o Learning through information inquiry 

o Learning from texts 

o Learning from recorded data 

6 . Material skills: Interacting with the physical world 

o Observation 
o Preservation 
o Fabrication 
o Intervention 
o Creation 




7 . Presentation skills: Communicating effectively and responsively 

o Self-presentation 
o Public speaking 
o Information distribution 
o Writing 
o Data illustration 
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8 . Qualification skills: Making distinctions and finding connections with words 

o Critical attention to language 
o Critical reasoning 

o Interpretive analysis 

o Interpretive synthesis 

o Creative thinking 

9. Quantification skills: Making distinctions and finding connections with numbers 

o Data measurement 

o Mathematical operations 
o Data analysis 

o Data synthesis 

o New approaches 

10. Technology skills: Using technology 

o Presentation technology 
o Information technology 
o Operation technology 

o Design and simulation technology 
o Experimentation technology 



The Development of Students' Skills 

The LS/PS program is designed to allow students to develop and polish their skills at their own rate; there is no 
expectation that any student will master all skills (nor that instructors will teach all skills). Each of the five sub- 
categories for the ten skills is arranged in a developmental model-that is, easier levels of the skill precede more 
difficult ones. 

The five levels are defined this way: 

1. Incidental occasion. These are short, often informal activities without attention to explicit standards, often 
accomplished while attending to something else. (This level of skill development is not assessed in the LS/ 
PS program.) 

2. Assessed effort. Here you will find short activities or assignments (for instance, a two- to five-page paper) 
with clear standards for acceptable quality. 

3. Substantial accomplishment. Here you will find extended projects of a larger scope, requiring more effort 
and attention to complete and stringent standards— a clear progression in ability beyond the level above. 

4. Broad ability. The competencies at this level increase in breadth, versatility, and reliability. 

5. Personal mastery. At this level, a skill becomes a part of who a student is and what he or she brings to any 
setting; a student’s colleagues vouch for him or her as a person with a deserved reputation for excellence at 
this skill. 

As an example, here are the five levels for the skill of Public Speaking (a subcategory of the Essential Skills of 
Presentation: Communicating clearly and responsively), with the least difficult level described first: 

Public Speaking 

o Incidental occasion: Speak before a group 
o Assessed effort: Prepare and deliver a short presentation 

o Substantial accomplishment: Prepare and deliver a substantial, persuasive presentation 
o Broad ability: Speak in a variety of organizational or public forums 

o Personal mastery: Deliver polished presentations consistently 
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How Does the LS/PS Program Work? 

The LS/PS program takes assessment beyond the level of measuring what students know to allowing them to 
demonstrate through class projects what they can do. The Skills Profile then allows students to show anyone outside 
of the school setting what they've learned. 

Participation in the LS/PS program is voluntary for students and teachers. Interested teachers identify the skills they 
are already teaching in their courses and work collaboratively to define the rubrics used to evaluate students’ 
achievement in each skill. Teachers enter a student's points in each skill area into a database that converts information 
from other courses into a graph that visually displays a Skills Profile in progress. Students can access their Skills Profile 
through the Internet and show it to whomever they choose. 



How Is the Skills Profile Used? 

We anticipate thatthe LS/PS Skills Profile will ultimately be of interest to students, faculty, counselors, administrators, 
transfer institutions, parents, and employers. The LS/PS Skills Profile and database help answer questions such as: 

o What skills have I developed? (for students) 
o What skills do my students bring to my course? (for faculty) 

o Which courses would help this student achieve his or her career goals? (for counselors) 
o How qualified is this applicant for the program he or she is applying for? (for transfer institutions) 
o Does this potential employee have the skills for this position? (for employers) 

How Does a Teacher Record a Student's Skill Development? 

An Internet-based system for entering LS/PS information has easy-to-use data entry forms on which teachers enter 
information about a student’s skills progress in a class. The system automatically updates a student's profile each time 
a teacher enters new information. 



How Are Students' Skills Assessed? 

Rubrics are the standards for evaluation-or assessment-that instructors at Inver Hills Community College have 
developed for many of the individual skills in the Essential Skills grid. Instructors from various disciplines developed 
the rubrics so that these standards could be applied to the assignments, tasks, or performances that students 
complete in any course or extracurricular activity. Each rubric defines four levels of performance, indicated by the 
numbers 1 through 4, with 4 being the best (essentially an A) and 1 being unacceptable (a D or an F). 



A Sample Rubric 

Here is a sample rubric from the Essential Skill of Collaboration: Working with People Effectively for the sub-category 
of Group Project at the Assessed Effort level: Collaborate to plan and deliver a one-time, short/small group project: 



4 


Attends 90 percent of group meetings; actively contributes ideas, seeks ideas from other group 
members, and integrates group ideas and suggestions into a coherent project design. Actively 
participates in the group’s creation, distribution, and completion of project tasks in both development 
and delivery stages. 


3 


Attends 90 percent of group meetings. Actively contributes ideas regarding the design and planning of 
the group project Performs all assigned tasks in both the development and delivery stages of the project 


2 


Attends 80 percent or less of group meetings. Actively contributes ideas regarding the project 
Performs most assigned tasks. 


1 


Attends 80 percent or less of group meetings with little, if any, active contribution to the group process. 
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How Are Rubrics Used? 

Instructors use rubrics to evaluate student performance and then record the points (4, 3, 2, or 1) on the Internet- 
based database that turns the scores recorded by all instructors into a Skills Profile for each student. The database 
allows instructors to enter scores quickly and to define the projects/assignments for which students earned skills 
points. 

Because the Skills Profile is designed to record the skills that students have mastered, only scores of 3 or 4 show up 
on a student’s Skills Profile. Instances in which they have performed at the 2 or 1 level are recorded in the database 
but do not show up on the Skills Profiles. 



Creating Rubrics 

Writing and refining rubrics is a collaborative process that involves instructors in sharing assignments with one 
another and discussing how they teach ano assess pyilicuiyi smmS. mic tuuiius liSicu uouw arc always subject tc 
revision and refinement. Because IHCC is a two-year college, instructors have focused on creating rubrics for only 
the first two levels of the Essential Skills grid-the Assessed Effort and Substantial Accomplishment levels-of 
performance most usually associated with the first two years of a college education. 



Does It Mean Extra Work for Teachers? 

Yes, initially. Adapting the rubrics for a course requires extra work, but once teachers begin using rubrics in class, 
the amount of work typically does not increase, and some teachers feel they spend less time grading because of the 
clear-cut criteria laid out in advance. 

Most teachers are surprised at how easily LS/PS fits into their current assignments. Rubrics simply clarify the criteria 
for assessing skills that instructors are already teaching. Also, IHCC has been able to make release time and/or 
stipends available to teachers who begin using the program. 



What Are the Benefits of Teaching an LS/PS Course? 

Developing rubrics for each skill area helps better define assignments. Students clearly see more the value On terms 
of skill development) in certain assignments. The cross-disciplinary workshops for faculty teaching the same skills 
in their courses create a new opportunity to interact with colleagues. Some previously unrelated courses have become 
linked. 



What Do Teachers Say about LS/PS? 

Most teachers participating in the program say that it ties in very well with their current assignments and actually 
makes grading easier by providing a well-defined set of criteria to show students. It has helped teachers clearly define 
for themselves which skills a particular course focuses on and consciously develop those skills as the course 
progresses. IHCC teachers who have participated in LS/PS are excited about the program: 

“Using LS/PS has made me a better teacher." (Speech teacher) 

“I like to be able to document outcomes of what I do.” (Psychology teacher) 

“This is a way to give students acknowledgment for the skills they develop in my course-what does an A or 
B in English 1108 rnean?" (English teacher) 

“It has made me more conscious of what is sufficient demonstration of competence of a project and the 
connection of projects to course outcomes.” (English teacher) 



O 
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“I’m not sure I’ve changed the way I teach, but I have redesigned my assignments so that they more clearly 
reflect what I’m trying to teach.” (Biology teacher) 
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How Did the LS/PS Program Develop at IHCC? 

IHCC has developed the LS/PS program from the bottom up over the past four years. In 1996, a group of IHCC faculty 
who had discussed ways to improve student learning brought forth the idea of adapting the Minnesota Skills Profile 
[MSP) developed by Dr. David Shupe and Jennifer Lundblad (both then atthe University of Minnesota) for use at IHCC. 
In 1997, 20 faculty and several administrators discussed and ultimately adopted the LS/PS concept as a tool to 
measure outcomes and improve student learning. During the year, faculty continued to meet in small and large groups 
to develop rubrics for teaching and assessing specific components of the ten Essential Skills. 

In the fall of 1 998, the LS/PS pilot program officially began with approximately 200 students participating. New faculty 
were invited to meetings and encouraged to participate in the program. By the fall of 2000, 30 faculty were teaching 
75 skills-enhanced courses with 1000 students participating. IHCC faculty have also made several presentations 
about the LS/PS program at national conferences. 

In the spring of 2000, IHCC was awarded two major grants from the MnSCU [Minnesota State Colleges & Universities) 
Emerging Curriculum and small grants from a private foundation. Funds were used to develop a web site, fund a 
summer workshop, and provide stipends/release time for faculty to fully implement the LS/PS skills into their courses 
and work with other campuses in the MnSCU system. In addition, IHCC was selected in August 2000 by the League 
for Innovation as one of 16 colleges to participate in the Twenty-first Century Learning Outcomes Project. 



Cheryl Frank is President of Inver Hills Community College in Inver Grove Heights, Minnesota. 

Joan Kuzma Costello is Vice-President for Academic Affairs and Student Development at Inver Hills Community College 
in Inver Grove Heights, Minnesota. 

David A. Shupe is Director of Academic Accountability, Minnesota State Colleges and Universities System. 
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Haven't We Done This Before? 
Developing and Implementing a 
Successful Outcomes Assessment Plan 
in a Multi-Campus System 



Timothy E. Dykstra 
Carol Brown 



History 

Oakland Community College COCC) is a thirty-five-year-old comprehensive institution serving Michigan’s Oakland 
County in the greater Detroit area. With more than 50,000 students served annually and instructional activity on five 
different campuses, it is the largest community college the state and the fourteenth largest in the nation. OCC has 
a rich history of assessment efforts dating back over thirty years. Some faculty, in fact, can produce assessment 
materials from 1 969. Forthe past decade, the college has made major efforts to engage in significant assessment efforts. 

However, despite literally thousands of hours of work by many highly dedicated and skilled faculty and staff members, 
the college lacked a sustained, institution-wide outcomes assessment model, and therefore sought to rectify the 
problem. In developing a strategic outcomes assessment plan, one of the first steps OCC undertook was to study why 
the college continued to lack a successful model even though there had been several attempts to build one. Three 
key problems with these past efforts emerged from that investigation: 

o It had not “closed the loops” by ensuring that the assessment information collected was actually used to 
improve the teaching/learning process; 

o The administration had not provided sufficient resources to sustain the project; and 

o The initiatives were allowed to evolve into localized, champion-specific efforts instead of a comprehensive 
institutional model. 

In response, the college developed a new approach to implementing a student outcomes assessment model. This 
model began by building on the successful elements of past efforts. It also included application of lessons learned 
from other recent large-scale initiatives such as implementation of an institutional technology plan and the adoption 
of a campus-wide academic master plan. Lessons from both failures and successes helped to shape the new effort. 



Challenges 

From the past outcomes assessment efforts of the 1990s and the most recent developments of a successful college 
model, several key challenges that large organizations face were revealed. Each of these issues had to be recognized 
and addressed in the revised conceptual framework. 

o Geography. With multiple campuses scattered throughout a metropolitan area, it was a challenge to get 
participation from all affected areas. 
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o Unions . The college has several vibrant labor unions and professional associations, each of which had 

contracts that needed to be respected. 

o Senior faculty . A significant percentage of the faculty were tenured, with more than twenty years of seniority. 

o Transient administration. At the same time, for the past several years, significant parts of the upper 
administration of the college had been transient, causing some in the college to question the need to accept 
new college initiatives since administrators were “here today and gone tomorrow.” 

o Confusion. Many faculty and administrators were unclearabout what student outcomes assessment really was 

and how it did or did not differ from traditionally held practices. 



Current Assessment Model 

Using the lessons learned from the past the college developed a new conceptual framework for outcomes 
assessment. In that development process, the college recognized it could not necessarily do all of the revising itself, 
so a strategic planning consultant was brought in to provide both expertise and an outside perspective to the issues 
at hand. The result of that development was a new framework that was actually based on and a modification of the 
earlier assessment efforts. Not all the work from the past was discarded. The new framework was built on three 
different piers, each closely connected to the other. 

First, given the college’s size and diversity, the assessment model had to have some degree of complexity itself. It 
called for assessment work to occur simultaneously on multiple levels: 

o Instructional 
o Discipline 
o Program 
o Institutional 
o Out-of-classroom 

These levels were necessary to encompass the many different kinds of assessment activity in the college, and they 
also enabled the model to tie to the elements of outcomes assessment that have appeared in recent discussions by 
NCA. The recent shift of focus by NCA called for the seamless weaving of assessment throughout the institution, from 
mission to planning to instruction. Given the college’s mission statement to be learner-centered, there was little 
difficulty in assimilating the NCA outline into the college’s framework. 

Second, the framework had to incorporate the importance the college placed on shared governance. Therefore, it had 
to recognize and use the college-wide infrastructure of academic committees. Given these political realities, this was 
the only way to gain widespread acceptance for student outcomes assessment. As with the writing of the assessment 
plan itself, this infrastructure incorporated both the old and new. For example, the entire outcomes assessment 
process was overseen by a steering committee with representation from major constituencies that included the 
leadership of the Faculty Association, College Senate, deans, and campus presidents. The previously existing Senate 
Student Outcomes Assessment Committee (SOAC] was preserved, and it was charged with providing the leadership 
for the implementation of the plan through the shared governance system. 

Finally, the revised model called for the creation of a smaller lead unit composed of deans and faculty coordinators 
from each campus. This unit was charged with responsibility for assisting the faculty in implementing the model at 
the operational level. This group of lead deans and faculty members worked closely with the political infrastructure 
described above and also provided grassroots assistance to individual faculty members as they undertook their 
campus assessment activities. 



What's Happened So Far 

The basic design for the assessment model was developed during the summer and early fall of 2000. Once the outline 
of the model was in place and introduced to the college community, the lead deans and faculty members got to work 
on a number of tasks. The tasks included: 
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1. Conducting an assessment inventory. Recognizing that many faculty members had been conducting outcomes 
assessment for some time, the college asked each discipline and academic department to supply a brief description 
of those activities. The request yielded a rich variety of activities, some of which are already consistent with the outlines 
of the expectations of the plan, others of which could be made to be so with some modifications. Often this 
modification was the addition of the feedback loop wherein the information gathered was put to use to strengthen 
teaching and learning activities in a course or program. A second benefit of the inventory was that it showed many 
faculty that they were much closer than they may have thought to participating in outcomes assessment 

2. Tying assessment to existing curriculum efforts. Whenever possible, the team of lead deans and faculty 
worked with appropriate committees in the college to incorporate outcomes assessment into existing curriculum 
structures. This allowed the college to minimize having to reinvent the wheel in developing the outcomes 
assessment processes. For example, in recent years the college had invested considerable time and effort in 
revising its general education framework. This led to ten new attributes that were the center of this framework. 
These attributes provided a listing of core characteristics for degree and certificate programs. As part of this new 
process, in the preparation for the review of a course’s attributes, faculty were expected to include appropriate 
outcomes assessment activities. 

3. Establishing outcomes assessment as a key institutional priority. The college’s administration identified 
outcomes assessment as one of the three strategic priorities for the college. Therefore, outcomes assessment 
became a major focus for the college’s Planning Council through its funding decisions. Outcomes assessment 
became one of the priorities in supporting new initiatives projects in addition to technology. 

4. Providing for sustained staff development related to outcomes assessment. The college sent a compre- 
hensive team to an assessment conference in Indianapolis, Indiana, in the fall semester. This team included not 
only lead deans and faculty coordinators, but also other faculty members from across the campuses. Both nay 
sayers and champions of outcomes assessment were part of this group. Recognizing the important role faculty 
must play in this process, the college agreed to double the number of faculty it had originally planned to send to 
the conference. 

The college has established a staff development plan that is designed to offer assistance on a sustained level. It 
addresses a variety of staff development needs that are importantto successful implementation. This plan focuses 
on a series of workshops to be held throughout the school year, and run by recognized outcomes assessment 
experts. Some workshops will focus on helping faculty conduct their assessment activities at the grassroots levels 
of their classrooms. Other workshops will focus on the ways department chairpersons, deans, and other academic 
leaders can support faculty and their assessment efforts. Still others are designed to address the broader vision 
for outcomes assessment and its larger implications for the teaching and learning process. 



Lessons Learned and Suggestions 

This major effort to revise the college’s outcomes assessment effort has yielded many lessons that might prove useful 
to other similar institutions. They are listed below in no particular order, for each one of them has proven to be vitally 
important. 

1 . Don’t reinvent the wheel. Build on what already exists in the organization. It does not even necessarily have to 
have been an assessment project for the pre-existing item to be useful to the assessment effort. For example, the 
OCC faculty coordinators saw that the college had recently been very successful in developing increased faculty 
awareness about how to incorporate technology into instruction by starting with small, informal groups of 
interested people and gradually building on them. In similar fashion, the faculty coordinators decided to host small, 
informal “teas" on each campus with other faculty members from that campus to discuss outcomes assessment 
issues and strategies. The hope is that these small gatherings will grow into college-wide discussions. 

2. Focus on faculty leadership for the assessment effort If it is to succeed, outcomes assessment must be a 
faculty driven enterprise. It should not be a top-down effort led by administrators. Instead, the administration 
should serve as a resource and support to the faculty members. 

3. Be sure to “listen to the voices.” In this focus on faculty, it is important for the leaders of the assessment effort, 
both the college’s administration and the faculty leadership, to listen carefully to ideas advanced by the faculty 
members and to be willing to modify existing plans. For example, at OCC many faculty members are saying, “Just 
give me the template to fill out, and I’ll give you what you want for the assessment project.” On the one hand, this 
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may be a dangerous request for a simplistic and ineffective form-completing ritual that falls far short of effective 
outcomes assessment. However, it can also bean important plea for some clear, straightforward guidelines faculty 
can use as they grow in their understanding of what outcomes assessment really involves. 

4. Appreciate the scale off the project For many large, multi-campus community colleges, adopting an effective 
and a substantive student outcomes assessment model involves a major cultural change for the entire institution. 
Despite what we might like to think about the adroitness with which community colleges can be flexible and can 
change, this particular cultural change demands considerable time, effort, money, and patience if it is to really 
happen rather than just appear to happen. And it is this kind of deep cultural change that is implied by the NCA 
guidelines for most colleges. 

5. Respect the role off effective communications throughout the effort. Keep the topic of outcomes assessment 
out in front of the organization. Don’t let it be a bright flash on the college’s horizon that quickly fades into the gray 
clouds of routine college functions. Instead, use visible signs and methods to remind people about outcomes 
assessment and how it impacts everyone’s work. Use small-group and large-group projects. Have a sustained, 
proactive attitude. 



Conclusion 

The revised student outcomes assessment model at OCC is still very much a work in progress. However, with faculty 
involvement, sustained administrative support, and effective staff development the prospects are bright for a 
successful outcome. 



Timothy £ Dykstra is Dean of Academic and Student Services at Oakland Community College in Waterford, Michigan. 

Carol Brown is Vice Chancellor of Academic and Student Services at Oakland Community College in Waterford [ 
Michigan. 
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Going Beyond the Data: 
Developing a Culture of Improvement 
with Part-Time Students 
and Adjunct Faculty 



Carol Scarafiotti 
Laura Helminski 

The Rio Salado College profile challenges even the most creative thinkers when it comes to conducting assessment 
of learning outcomes and making subsequent improvements in teaching and learning. Rio Salado College, one of the 
ten Maricopa Community Colleges in Phoenix, Arizona, is a comprehensive community college that specializes in 
three areas: customized and unique programs and partnerships, accelerated formats, and distance learning. It is 
immensely successful, growing at a rate of 1 5 percent for the last five years, to the extent that it is now the third largest 
of the ten community colleges. However, it does not have a campus; most of its students engage in course work at 
corporate sites or in their homes. It has 680 adjunct faculty members and 21 residential full-time faculty members, 
most of whom serve as faculty chairs. It serves approximately 26,000 students annually, with 44 percent of these 
students enrolled in 1 -4 credits each semester; 50 percent of the students are new each semester. Approximately 1 75 
students receive an associate degree each year, while 4600 students receive occupational certificates. Furthermore, 
Rio Salado College’s distance learning program has rolling enrollment, allowing students to begin most of its 300 
courses every two weeks. 

In spite of the very nontraditional nature of the college, the educational leaders at Rio Salado College are committed 
to assessment of student learning and to improvement of the teaching and learning processes. Over the years, we 
have developed expertise in customizing the assessment process to our unique college profile. In this paper, we will 
reveal some insights we have gained through our experiences that relate both to the design and the implementation 
of a plan for assessment of general education competencies as well as to the improvement of learning. 



Dealing with the Reality of the College Profile 

Dealing with the reality of the college profile in the design of an assessment plan meant that we would craft the 
assessment design and processes to fit our unique institution. Our first realization was that we would be remiss if we 
focused our assessment efforts only on degree-seeking students, since the majority of our students are part-time 
students who may never receive a degree from Rio. So we designed our assessment plan to evaluate the reading, 
writing, problem solving, and critical thinking skills of all students, both degree and non-degree seeking, and then 
designed a more extensive assessment for those who actually receive degrees from Rio. 




A more in-depth view of the college student profile also helped us realize that within the two large groups of degree 
and non-degree seeking students were distinct cohorts that we needed to consider. For example, we had a large 
cohort of high school students classified as “able and ambitious” who enrolled in 1 5-20 credit hours of math, science, 
and languages over a two-year period. The reality is that these students will receive their degrees from various 
universities, not from Rio Salado College. But it was important for us to understand how well these high school 
students performed on college level competency assessments. Likewise, since writing is an important skill for 
students in our distance learning courses, we wanted to understand how well distance learning students wrote. So 
our assessment design included evaluating the learning of three cohorts: the able and ambitious students, those 
taking distance learning courses, and those enrolled in our in-person/accelerated classes. 
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Our understanding of the three cohorts also aided us in the design of the assessment implementation. For example, 
assessments of distance learning students were conducted online for those enrolled in Internet courses. For students 
taking in-person classes, we either administered the assessments during the class period or mailed the assessments 
directly to students. 



Determining What We Wanted to Know About Part-Time Students 

Over time, we learned how critical it was to explore thoroughly what questions we wanted to answer about our 
students’ learning, but we also discovered that this learning came with experience. Initially, we merely wanted to 
establish a baseline that told us by cohort how all students were performing in competency assessments of writing, 
reading, problem solving, and creative thinking. After receiving our first set of assessment data, we decided that we 
wanted to look at the “value added" question: Does taking more general education courses at Rio result in higher 
assessment scores? Later we wanted to compare Rio’s student assessment outcomes with student assessment 
outcomes from similar colleges. Then, after two years of assessment, we wanted to knowthe correlation of a student’s 
assessment scores with performance in class. 



Knowing the Imperfections of the Plan and Working with Them 

In addition, we discovered that assessment of part-time students is messy and that it is better to have a workable 
design and plan with known imperfections than it is to hold out for a perfect plan that never happens. For example, 
we decided early on that we could not assess every part-time student, even though we knew our data would be better 
if we did. As a result, our assessments outcomes are based on sample data. However, when we realized that our 
sample would not be completely representative of the desired population group, we documented the situation so that 
it would be accounted for in the analysis of the data. We then vowed to improve the sample for the next year. 



Recognizing the Need for Convenience 

Part-time students and adjunct faculty members have several characteristics in common; they are busy, pragmatic, 
and watchful of their limited time. Therefore, convenience must be a main consideration when they are involved. As 
a case in point, distance learning students take their assessments in a distance learning format. Adjunct faculty 
members teaching at remote sites who are involved in the distribution and collection of assessments receive packets 
with easy-to-follow instructions that can be mailed back to the college in pre-stamped mailers. We also have couriers 
deliver and retrieve some of the assessment packets. 

If we knew that the assessment process would inconvenience a student or faculty member, we were willing to pay 
for their participation. That is, when we wanted students to come to Rio to take a five-hour standardized examination 
that was not class-related, we paid them to do so. 



Including Adjunct Faculty Members in the Planning of the Assessment Design 

Adjunct faculty members offer us invaluable insight into the design and implementation process because they often 
understand better than we the needs of adjunct faculty members and part-time students. They participate in the design 
of the faculty-developed program level assessment instruments and department level exams, in data analysis discussions 
in their departments, and in design of interventions and improvements in curriculum and teaching and learning strategies. 

In August 2000, more than 200 adjunct faculty members attended the Fall Semester All Faculty Meeting and 
participated in a dialogue activity on assessment. Students also joined in discussions that answered the questions: 
[1) How can we increase the incidence of college level skills being used in student classwork? [2) What is the faculty 
member’s responsibility for expecting and requiring college level skills to be used? [3) Given that we want to set high 
expectations, what are some barriers to making this happen? 



Removing the Fear Factor from Assessment by Focusing on Improvement of Student 
Learning 

Allot our faculty members appreciate Rio’s philosophy that assessment data should be “fear free.” Data are not used 
to judge faculty performance; instead, they are used to improve the curriculum and to improve teaching and learning. 
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The data help faculty chairs and adjunct faculty members focus discussions on curriculum, course lesson 
assignments, textbook selection, department exams, and specific teaching and learning strategies for specific student 
cohorts. 

Adjunct faculty members see value in the assessment process when they are aware of its direct application to 
improving teaching and learning. Therefore, faculty chairs take pride in showing adjunct faculty members specific 
improvements that have been made in certain distance learning courses because of the assessment data. 



Evolving into a Culture of Improvement 

The college leadership recognizes that it is important to develop a college-wide culture of improvement that extends 
to all students and adjunct faculty members, but also knows that it takes a major commitment to do so. Consequently, 
the college leadership (with input from adjunct faculty members and students] has developed and is implementing 
a plan for developing a culture of improvement. Phase I of this plan includes a college-wide focus on a “big gesture” 
as a way of increasing attention to assessment and improvement. In Phase I, the student achievement committee 
selected the Rio writing competency as the emphasis for the 2000-2001 academic year, and the faculty chairs the 
cabinet approved it as a priority goal for the next several years. As a result of this goal, expectations for college level 
writing are stated in all course syllabi and in writing assignments. Students are given feedback about their ability to 
demonstrate writing above or below the college level writing criteria. The writing competency coordinator works with 
all of the faculty chairs to help increase students’ writing skills in all disciplines. 

Activities and training are underway for operationalizing other parts of the plan to develop a culture of improvement. 
Clearly stated and visible expectations and standards, shared values and vision, and high levels of involvement across 
the college are the basis for work on assessing and improving student learning. The faculty, administration, and staff 
are focused on understanding and cultivating this culture and seek the challenge to model those behaviors that are 
important to a culture of improvement. More information on developing a culture of improvement will be highlighted 
in our presentation at the 2001 NCA Annual Meeting. 



Carol Scarafiotti is Dean of Instruction at Rio Salado College in Tempe, Arizona . 

Laura Helminski is Communication/Reading Faculty Chair at Rio Salado College in Tempe, Arizona. 
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Stairway to Heaven: 
Continual Incremental Change 
in the Assessment Process at a 
Rural Community College 



Adrian P. Hunt 
Melissa L. Shaffer 
Nathan T. Allen 
Natalie L. Gillard 

The assessment of student academic achievement is an evolutionary process that proceeds incrementally. Mesa 
Technical College is a small, rural community college that has been incrementally upgrading up its assessment 
process for the past five years. This continual, step-wise evolution has resulted in an integrated, multi-faceted 
assessment process embedded throughout the institution. Current assessment efforts utilize assessment at all levels 
of the institution (classroom, program, and institutional) and all stages of the student’s academic progress [prior to 
registration, within semester, end of program, after graduation) and utilize both direct and indirect measures of 
student learning. The assessment process was singled out as one of the strengths of the institution by the NCA site 
team in 1999. This paper will emphasize the philosophical underpinnings of this developmental process as well as 
the practicalities of its implementation. 



Mesa Technical College 

Mesa Technical College was formalized in 1994 and evolved from Tucumcari Area Vocational School, whose roots 
go backtothe New Mexico Vocational School Act of 1 979. The college is one of three technical colleges in NewMexico 
and offers the Associate of Arts and the Associate of Applied Science degrees. Several degree programs offer 
certificates and diplomas. In addition, there is a certificate program in Commercial Truck Driving. 

The college has a broad range of general education and technical programs as well as three distinctive ones-Farrier 
Science, Fine Arts/Bronze Foundry, and Natural Sciences-that have attracted students from more than a dozen states 
and several foreign countries [e.g., Australia, Belize, Israel, Germany, South Africa, and Brazil). 

The institution had a fall 2000 enrollment of just over 500, with a diverse student body of whom 56.3 percent are female 
and 48.4 percent are Hispanic or represent other minorities. This is a good reflection of the multicultural character 
of Quay County, the population of which is 40 percent minority. 

Mesa Technical College is situated in Tucumcari, New Mexico, a rural center with a population of just under 6700 
within a county of about 1 0,000 people. The local economy is dominated by tourism and agriculture. 



Philosophy of Assessment Process 

The Student Learning Assessment Committee considered several options for an overall conceptual model for the 
assessment of student academic achievement. After extensive research, the committee decided to recommend the 
usage of the Input-Environment-Outcome 0-E-O) model as expounded by Alexander Astin in his book, Assessment 
for Excellence [1991). 
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Astin’s model emphasizes the necessity of consideration of what the student brings to the course/program, the 
environment of learning within the course/program, and studentoutcomes.These three elements are interdependent, 
and assessment of student academic achievement cannot be worthwhile without consideration of all three. The 
current Student Learning Assessment Model at Mesa Technical College involves a broad range of assessment 
measures, both direct and indirect, that are utilized at the classroom, program, and institutional levels and at all stages 
of the student’s academic progress (Table 1). 



Small Is Beautiful 

The small size of Mesa Technical College is a tremendous advantage when it comes to planning and implementing 
assessment. The entire full-time faculty of 1 3 can sit around a single table and decide on substantive changes, and 
these can be implemented almost immediately. This ability to '‘turn on a dime” has allowed Mesa Technical College 
to adapt quickly in its assessment initiative. 



Incremental Evolution of Assessment Process 

Mesa Technical College is committed to the premise that the assessment initiative must continually evolve to flourish. 
Thus, the Student Learning Assessment Committee is constantly involved in a dialog with all the constituencies of 
the college to stimulate feedback-driven changes, for example, by holding joint meetings with the Faculty Council 
every semester. 

It is paramount to the success of the assessment process that there are both feedback loops and incentives for faculty 
to participate. The Student Learning Assessment Committee provides a number of documents that facilitate feedback 
loops, including semester and annual reports on the status of assessment at the college. The annual report of the 
Student Learning Assessment Committee is presented by the Dean of Instructional Services [a member of the 
committee) to the President, who forwards it to the Board of T rustees, who then review it at one of their regular public 
meetings. The Student Learning Assessment Committee also distributes a regular newsletter called Assessment News 
that not only provides information, but also feedback from faculty. Incentives are as important as feedback, and each 
semester at least one award for excellence in implementing assessment is presented by the Student Learning 
Assessment Committee. Lists of award recipients are published in Assessment A/ewsand forwarded to the President 
to be presented to the Board of Trustees. 

This process of feedback and adjustment has led to continual, incremental change and refinement in all levels of 
assessment at the institution (Table 2). For example, within the classroom environment, assessment strategies were 
first integrated in 1 996. Initially, all faculty were encouraged to try Classroom Assessment Techniques [CATs) and pre- 
test/post-tests in one of their classes in the fall of 1996. This was such a success that the faculty voted to use three 
CATs and a pre-test/post-test in each course in the spring of 1997. Subsequently, there were refinements in these 
numbers so that all courses utilize one CAT per credit up to three, and innovative CATs were developed for 
nontraditional education such as distance learning and internships. Every semester, the Student Learning Assess- 
ment Committee analyzes feedback from assessment and proposes new changes. If these changes are ratified by the 
Faculty Council, they are adopted immediately, a luxury made possible by the small size of the college. Similar review 
and refinement occurs with respect to program- and institutional-level assessment. 

Some changes in the assessment initiative have amounted to major paradigm shifts. Such exciting changes happened 
during the fall 2000 revision of the Student Learning Assessment Model. First, the committee suggested an 
adjustment to the college’s mission and goals to include wording on student learning, which was ultimately ratified 
by the Board of Trustees. Second, the model was significantly revised to emphasize feedback-driven changes as 
opposed to the collection of numerical data on assessment. Third, the Student Learning Assessment Committee 
changed the name of the model from the Student Outcomes Assessment Model to the Student Learning Assessment 
Model. This reflected the committee’s view that the word outcomes suggested an undue emphasis on product as 
opposed to process. Assessment is involved with all aspects of the learning experience, and the Student Learning 
Assessment Committee felt that the word learning should be substituted in the name of the model. 



Maturing Toward a Learning Community 

The assessment initiative at Mesa Technical College continues to evolve and has now developed into a meaningful 
and pervasive assessment process that is instilled into the institution fabric. Twelve defining characteristics 
demonstrate the effectiveness of assessment at the college: 
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1 . Commitment to assessment is demonstrated by many factors, including a Board of Trustees statement in favor 
of assessment and the establishment of the Student Learning Assessment Committee as a standing 
committee of the college, the establishment of a budget line item for assessment, and many others. 

2. Mesa Technical College has numerous explicit expressions of its commitment to assessment, including 
sections in each revision of the college catalog, faculty handbook, and student handbook. 

3. The college utilizes multiple measures of assessment, including direct and indirect measures of learning. 

4. Assessment is implemented at the classroom, program, and institutional levels; in fact there is assessment at 
all institutional levels. 

5. There is assessment at all academic stages of the student’s advancement. 

6. Measurable objectives are in place for every course taught at the college, and measurable program objectives 
are published in the college catalog. 

7. Numerous methods of feedback are a part of the assessment process at Mesa Technical College, including 
joint meetings of the Faculty Council and Student Learning Assessment Committee every semester. 

8. Incentives are in place to encourage faculty to buy into assessment, including awards for outstanding 
proponents of assessment every year. 

9. Data dissemination is a major goal of the Student Learning Assessment Committee and is accomplished 
through such vehicles as Assessment News. 

1 0. Continuous progress in the assessment process is demonstrated by the numerous refinements that have been 
adopted since 1997. 

11. The Student Learning Assessment Committee is dedicated to continuous refinement of the assessment 
process not just through annual reviews of the model, but also through changes to forms and procedures 
almost every semester, as documented in the semester reports of the committee. 

1 2. Change as a result of assessment is central to successful programs and is demonstrated at the college in every 
Faculty and Program Assessment Outcomes Form submitted each semester. 



As Mesa Technical College continues its journey toward a learning community, revision of the assessment process 
is the norm. Climbing the stairway to Heaven requires continual feedback-driven change. Are we there yet, Dad? No, 
but we are always getting closer. 



Adrian P. Hunt is Chair of the Student Learning Assessment Committee, Museum Curator, and Natural Sciences Faculty 
at Mesa Technical College in Tucumcari, New Mexico. 

Melissa L. Shaffer is Dean of Student Services at Mesa Technical College in Tucumcari, New Mexico. 

Nathan T. Allen is Division Chair for Applied Science and Animal Science Faculty at Mesa Technical College in Tucumcari, 
New Mexico. 

Natalie L Gillard is Division Chair for Arts and Sciences and Business Administration Faculty at Mesa Technical College 
in Tucumcari ; New Mexico. 



Table 1. Mesa Technical College Assessment Matrix 2000-2001 
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Table 2. Evolution of Classroom-Level Assessment at Mesa Technical Col lege 
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Developing and Implementing a 
Process for Continuous Development 
and Improvement of Graduate Programs: 

A Case Study 

David Overbye 
Romuald Stone 

Introduction 

A fundamental mission of any institution of higher education should be to place a strong emphasis on continuous 
improvement of its programs. A key determinant of success in any graduate degree program is the ability to 
continually assess and improve program quality and effectiveness. Regular academic program reviews represent one 
mechanism for achieving these goals and objectives. 

At the end of 1998, Keller Graduate School of Management (KGSM) committed to conducting a formal self- 
assessment and evaluation of all its academic programs over the next five years. KGSM currently offers six master’s 
degree programs. We decided to start with the Masters in Human Resource Management (MHRM] program review, 
which was completed in June 1 999. This process allowed us to identify strengths and weaknesses in our program and 
to target areas of improvement. As a result of this review, we feel our MHRM program will become stronger and will 
better meet the needs of our students as the recommendations of the self-assessment are implemented. It is 
interesting to note that subsequent reviews of the MHRM program self-study received glowing praise from numerous 
evaluators. 



A Brief History of the Institution 

Keller Graduate School of Management was founded in Chicago in 1973 on the idea that the most important 
components of management education are effective teaching and student mastery of practical management skills. 
The first class consisted of seven full-time students. By the late seventies, most of the school’s nine hundred students 
were pursuing their MBA degrees through an evening program introduced in 1 974 to serve working adults seeking 
additional professional business education. 

To address its students’ special needs and in light of industry shifts within the Chicago metropolitan area, Keller 
opened four suburban centers between 1975 and 1980. The first center outside the Chicago area opened in 
Milwaukee in 1983. 

In 1987, Keller acquired the DeVry Institute system, providing an impetus for the school to broaden its geographic 
operations. Today, Keller offers its practitioner-oriented curricula in thirty-nine locations coast to coast as well as 
through its Online Education Center. 

Keller’s MBA degree was augmented in 1991 when the Master of Project Management (MPM] program was first 
offered. The focus was broadened in 1993 with the introduction of the Master of Human Resource Management 
degree. In 1997, a new Telecommunications Management program was first offered, and last year the two newest 
degree programs— Accounting and Financial Management and Information Systems Management-became available. 
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Becker CPA Review, acquired in 1996, complements the school’s growing range of educational services. Becker is 
the world leader in providing preparatory coursework for the Certified Public Accountant and Certified Management 
Accountant examinations. Becker courses are offered in forty states plus the District of Columbia, and in eighteen 
foreign countries. It was recently fortified with the addition of Conviser-Duffy CPA Review, and now does business 
as Becker-Conviser CPA Review. 



Overview of the MHRM Program at KGSM 

The Master of Human Resource Management program prepares students to be more productive in their organizations 
by teaching concepts and skills needed to plan, control, and direct organizational requirements for effective and 
efficient use of human resources. As noted earlier, this program was added to the curriculum in 1993. Coursework 
is taught from the practitioner’s perspective and focuses on applying human resource concepts and skills to real- 
world problems and opportunities. 

MHRM students must complete fifteen courses, including five management core and seven program-specific 
courses— one of which is a capstone course in human resource planning— in key human resource functionai areas. 
Three elective courses must also be completed. 



The Purpose of the Program Review 

For almost the first twenty years of Keller’s existence, the only program the school offered was the MBA. Then over 
a relatively short period of time, the school added five more graduate degree programs. Prior to 1991, the school relied 
mainly on an informal process of curriculum and program review that involved faculty working with the Vice President 
for Academic Affairs. 

The first comprehensive review of the MBA program was conducted in 1994; however, no formal documentation 
detailing the process or changes was produced. The Master’s in Project Management program has been revised on 
two occasions since 1991, but again no formal document of that process was produced. 

The MHRM program represents the first formally documented program review. As the school grew larger in terms 
of students and programs, we found ourselves operating in more jurisdictions. Consequently, we find ourselves having 
to explain and defend what we do much more than in the past. Hence, future program reviews will have formal reports 
that are concise and readable by various reviewers. 

Given the previous history and the school’s emphasis on continual institutional improvement, we embarked on a 
formal process for conducting a self-assessment ofourgraduateprogramsthatwould serve the following institutional 
purposes: 

o To establish a formal process for continuous improvement and review of existing curricula; 

o To develop a working framework for understanding and improving the school’s degree programs; 

o To review program goals, objectives, and performance trends and relate them to the school’s mission and 
goals; 

o To conduct a comprehensive, reliable evaluation of the school's degree programs as a solid foundation for 
institutional planning and improvement; 

o To assess satisfaction of students, alumni, and faculty with the respective program; 

o To respond to the key needs and expectations of our current students, alumni, and stakeholders to ensure the 
relevance of the school’s educational programs and objectives; 

o To develop an action plan that addresses key short- and longer-term program improvements; and 

To strengthen the school’s culture in which assessment of academic programs is a perpetual spiral of 
planning, implementing, assessing outcomes, and acting on feedback from our stakeholders to continually 
improve all aspects of the school. 

n 
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The Program Review Process 

A major objective of the preview of the MHRM program was the development of a consistent model that could be 
used in future program reviews. We thus had as a goal both reviewing the MHRM program and reviewing it in such 
a way that the review process followed could become a template for future reviews of other programs. Administra- 
tively, the responsibilityforthe program review was vested inthe MHRM Program Manager, reporting to the Director 
of Curriculum. 

Keller identified several groups of program stakeholders whom we viewed as critical. These critical stakeholders were 
the faculty teaching in the program, the employers of our graduates, and the professional societies that had as an 
objective the improvement of practice in the field. Faculty were involved in the process via extensive consultation; 
employers were involved via focus groups and surveys. Professional societies were involved via participation in their 
meetings and through examining the work they had done to define a common body of knowledge that presented the 
topic and skill areas needed for successful practice in the human resources field. We also conducted a comprehensive 
review of the research literature to determine the current thinking on the conceptual framework underlying the 
professional practice of human resources at the graduate level and how academic programs should best support that 
practice. 

The conceptual framework that we developed substantiated and validated the basic foundation of our program 
structure, staffing, resources, assessment procedures and outcomes. However in performing a gap analysis, we did 
determine that in several areas the field had evolved in a direction that mandated changes to our program. One 
example of this is the increasing use of technology in the human resources field. The gap analysis between our existing 
program structure and the results of our analysis of current professional practice requirements became the basis for 
program change recommendations. 

The final step in the review process was the preparation of a report that would serve both as a basis for the 
communication of program change recommendations and as a model that could be followed by other program 
managers in subsequent program reviews. 1 This report also became the basis for presenting the results of the review 
to the school’s academic strategy group chaired by the dean. The presentation resulted in a useful discussion among 
the strategy group members, and ultimately approval for the majority of proposed recommendations to be 
implemented in the next academic year. 



The Keller Program Review Model 

The structure for our program reviews is based on the format found in Figure 1. The various sections were developed 
by using similar topic areas found in an institutional self-study as well as our own determination of what information 
would be beneficial for this program review. 

Our presentation will center on providing an overview of each of the major sections of the study. 

In brief, our study reviewed the history of the MHRM program, its goals, objectives, and structure. We also examined 
the institutional program objectives and how the MHRM program specifically strives to meet those goals. 

A self-study and review of the MHRM program would be incomplete without a comprehensive review of the literature 
to capture the latest thinking on what university programs must offer to prepare human resource professionals for 
the future. To this end, we identified nine key sources of information and research that helped us determine which 
topics and competencies are the most important to include in a human resource management curriculum. 

We next examined program performance and structure. This included an overview of the curriculum planning process, 
a comparative program analysis to see how our MHRM program stacks up to other programs in terms of functional 
HR competencies. We reviewed data related to program trends-enrollment history, course takers, and sections of 
courses offered per term. 

Academic staffing issues are also addressed in the self-study. Topics include faculty credentials and skills, high- 
quality teaching, teaching evaluations and faculty performance ratings, faculty selection and training, and faculty 
meetings. 
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We also documented the extent to which the learning resources, technology, library, and institutional materials are 
sufficient to meet the educational objectives of the school. PR A 
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Program evaluation was an important component of our study, in particular program effectiveness. We conducted 
an extensive student and alumni survey in March-April 2000 to assess student satisfaction with the program. Input 
from a faculty focus group was also included. 

As a result of this comprehensive self-study, it was apparent that the MHRM program could be strengthened and 
improved in several areas. Five short-term recommendations to improve the MHRM curriculum and a long-term 
strategic plan with key goals were developed for the continual improvement of the program. 



Conclusion 

Regular program review is a critical component of any academic institution's efforts at continuous improvement. 
A framework for developing such a review for a specific degree program was presented. As a result of the program 
review model developed at Keller Graduate School, coupled with the involvement of key constituents in the process, 
we believe we now have a solid process for ensuring continued review and improvement of our academic programs. 



Notes 

1 . A complete copy of the 1 30-page report is available to interested parties. Please send an email requesting a copy 
to rstone@keller.edu. 



David Overbye is Director of Curriculum at Keller Graduate School of Management in Oakbrook Terrace , Illinois. 
Romuald Stone is MHRM Program Manager at Keller Graduate School of Management in McLean , Virginia. 
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Figure 1. Format for MHRM Program Review Report 



Executive Summary 

Section 1: Background 

o History of the Program 
o Goals and Objectives 
o MHRM Program Structure and Content 
o Institutional Program Objectives 

o Consistency with Mission and Resources: A Summary Statement About the MHRM Program 

Section 2: Literature Review 

Section 3: Program Performance and Structure 

o Curriculum Planning, Evaluation, and Revision Process 
o Program Comparison 
o Program Trends 

Section 4: Academic Staffing 

o Faculty Credentials and Skills 
o Effective Creation and Delivery of Instruction 
o Teaching Evaluation and Effectiveness 
o Faculty Performance Ratings 
o Faculty Selection and Training 
o Faculty Meetings 

Section 5: Academic Resources 

o Adequacy and Quality of Curriculum Guides 
o Technology in the Classroom 
o Library 

o Professional Librarian 
o Applied Business Research 

Section 6: Evaluation 

o Monitoring of Program for Effectiveness 
o Assessment of Student Learning Outcomes 
o Student Satisfaction and Surveys 
o Current Students 
o Alumni 

o Faculty Focus Group 
o External Review 
o Program Strengths 
o Program Weaknesses 

Section 7: Recommendations 

o Changes to MHRM Program 
o Strategic Plan: 2001-2006 

Appendices 

o Appendix A: Content Outline of the HR Body of Knowledge 
o Appendix B: Student Survey Form 
o Appendix C: Alumni Survey Form 
o Appendix D: Terminal Course Objectives for HR Courses 
o Appendix E: Summary of Student Written Comments 
o Appendix F: Summary of Alumni Written Comments 
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Musing Upon Our MUWSE: 
Identifying and Overcoming Obstacles 
Via a Model of Program Assessment 



introduction 

Faculty members in English at the Metropolitan Community Colleges of Kansas City (MCC) have developed a 
programmatic writing assessment tool dubbed MUWSE [Multiple Writing Samples in English), this tool will be used 
as a model for discussing the kjnds of obstacles that institutions can face in piloting assessment projects, in finding 
the means around those obstacles, and in utilizing the process as well as the results of assessment to change teaching 
and learning. 

After discussing this particular example, audience members will be &sked to complete a worksheet identifying 
problems with assessment in their own institutions, as well as any solutions they’ve developed so far. This worksheet 
will be the basis for small group discussions and an opportunity for feedback to the entire group. 

Background of MCC 

Our district faces both practical and academic obstacles to assessment. MCC is a system of four colleges (Blue River, 
Longview, Maple Woods, and Penn Valley) that serves the urban, suburban, and rural areas on the Missouri side of 
Kansas City. These colleges range in size from 3000 to 9000 students, and each college serves a large proportion of 
nontraditiona! students. Each college also provides outreach for even more specialized groups of students, including 
off-campus classes, distance learning environments, and special needs students. 

In addition to the diversity of the student body, the faculty teaching English 1 02 use a variety of approaches to writing, 
from writing across the disciplines to a pragmatic linguistics orientation to courses in argumentation. ComrrlUnication 
among faculty at the four schools is difficult, since the schools are a half-hour to an hour’s drive away, and faculty 
members are frequently torn by other classroom and committee responsibilities that make even the scheduling of 
meetings problematic. While faculty have access to telecommunication centers for video conferencing, generally 
faculty schedules are so tight that teleconferencing becomes unfeasible. 

The Process 

The English faculty members at MCC have been engaged in programmatic.assessment of the exit competencies of 
Composition and Reading II [English 102) students for the last three-and-a-half years. The assessment asked faculty 
to collect two or three final drafts of writing produced in their English 102 classes. Names were removed from these 
papers and replaced with Social Security numbers to preserve anonymity. Scorers were required to attend ninety- 
minute norming sessions to acclimate them to the rubric; then the packet of writings from each student was scored 
holistically on a scale from 1 to 6 by two separate readers, with discrepancies of two points or more going to a third 
reader. The data collected were reported to the district’s English faculty at an annual discipline meeting at in-service, 
and data were also made available to faculty and administration in a printed publication. The preliminary results for 
the most recent reading show that the average MCC score for MUWSE was 4.0, “competent” by our standards. 



J. Anne Dvorak 



Mike Warren 
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Obstacles Before the Project 

In the first year, we determined that the project was feasible, but there were philosophical and theoretical problems 
with identifying the project as a “portfolio'’ sampling, since our sampling would not fit the most widely accepted 
definition of a writing portfolio currently used in composition theory. In addition, many faculty were concerned about 
anonymity and the uses of the data collected, and others were resistant to the idea of being required to incorporate 
portfolios into their English 102 classes. Faculty members also wanted to make sure that the assessment tool used 
was fair and useful for their student population. In the second year, we revised the project to collect multiple samples. 
In the third year, the faculty problems became manifest with concerns about the burden that data collection would 
present to adjunct faculty and with ensuring enough faculty buy-in to collect sufficient data and enough scorers to 
assess these data. 



Current Obstacles 

As the project has continued, the following remain or have become areas of concern: 

o Data collection as a burden to faculty, especially adjunct faculty 
o Logistics of communication, faculty involvement, and data collection 
o Lack of faculty participation in scoring sessions 
o Faculty burnout on the notion of assessment 

o Problems with disseminating results and internalizing the data to change the program 
o Inter-rater reliability. 



Discussion of the Obstacles 

The biggest discrepancy in moving from the pilot project to having a full data sample to assess has been the problem 
of inter-rater reliability, which has been low [.02] in the latest data we have analyzed. There are a number of possible 
factors that may have caused this problem. 

o The increase in number of scorers 
o The change in rubrics 
o Reader fatigue 

o Time elapsed between first, second, and third readings. 

We are considering changes in norming and initiating discussions of readers’ expectations for English essays to try 
to compensate for this problem. 



Overcoming Obstacles 

With time, persistence, and more publicity, the problems with communication, involvement, and data collection are 
abating. We formed a committee that allowed representation from all four schools, and these representatives are 
responsible for collecting data and reporting any problems from their colleges back to the committee. This helped 
solve the faculty buy-in problem, and removed some of the burden of decision-making and problem-solving 
responsibility from the chair. 

Because the test was locally developed, it was easierforfaculty engaged in the reading process to determine problems 
with our program and to start coming up with interventions even before the data had been analyzed. Faculty at one 
college who went through the scoring session have already recommended and changed the text required for 
English 102, moving from a discipline-based approach to an argumentation rhetoric to emphasize critical thinking 
skills more overtly. 
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Next Steps 

Currently, we are trying to finish off a year's worth of sampling and engage in another faculty in-service discussion 
of what we have learned, both from the results and the process. We need to re-think both the norming and recruiting 
process for scorers in orderto improve involvement and inter-rater reliability. Those involved in the reading earmarked 
the need for changes in textbooks, in writing assignments, and in the emphasis on and expectations for 
documentation within 102 classes. We need to disseminate this information more widely within MCC, especially to 
other disciplines as we continue to expand our Writing Across the Curriculum projects. The results of this assessment 
will help faculty across disciplines understand and anticipate the special needs of MCC writing students. Finally, we 
hope to ensure that future hires are knowledgeable about assessment and are willing to participate in the assessment 
process. 



Workshop Agenda 
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o Small group discussion of problems and solutions 
o Feedback. 



J. Anne Dvorak is Faculty, English, at Longview Community College in Lee's Summit, Missouri. 
Mike Warren is Faculty, English, at Maple Woods Community College in Lee's Summit, Missouri. 
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General Education Assessment: 

A Course-Embedded Assessment Model 



Sandra Flake 
James Doerner 
Richard Trahan 



Introduction 

The University of Northern Colorado has developed and adopted an assessment model for general education that 
assesses outcomes at the course level of the program. This course-embedded assessment model was piloted by two 
faculty members, one of whom chaired the team that recommended the model, and it is currently being used by 
approximately 20 faculty across the general education program in both skills courses and content courses. The model 
should be fully implemented by the end of the 2001-2002 academic year. 



Context 

The University of Northern Colorado is a general baccalaureate and specialized graduate research university, which 
will be classified as a doctoral research-intensive institution under the new Carnegie classification [formerly 
Doctoral I). The university enrolls more than 10,000 students and provides a wide array of undergraduate programs 
in five colleges: Arts and Sciences, Education, Health and Human Sciences, Monfort College of Business, and 
Performing and Visual Arts. Master’s and focused doctoral programs, often in applied and professional areas, are 
offered in all of the colleges except the Monfort College of Business. The historic mission of Northern Colorado has 
focused on teacher preparation, with 2500 students currently enrolled in teacher preparation programs. In addition, 
several of the graduate programs reflect the legislative mandate that Northern Colorado provide statewide leadership 
in professional education. 

The present general education program at Northern Colorado has been in place since the late 1980s and is a 
university-wide program of 40 semestercredits-1 0 credits in skills courses in composition, mathematics, and physical 
activity; and 30 credits in the following content areas: arts and letters, social sciences, science and mathematics, and 
interdisciplinary, multicultural, and international studies. The program is the responsibility of the Dean of the College 
of Arts and Sciences and is governed by a committee of the Faculty Senate, the General Education Council, which 
has representation from all colleges and from the three skills and four content areas within the program. The Dean 
of Arts and Sciences serves as a non-voting member of the council. 



Limitations of Typical General Education Assessment Models 

Many general education assessment programs rely on standardized testing of general knowledge. While standard- 
ized tests have the advantages of ease of administration, comparative results from one testing administration to 
another as well as among tested institutions, and global measures that relate to most general education curricula, 
there are several limitations to using standardized tests as primary methods of assessment. 
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1 . Unclear relationships between the program learning outcomes and the test Depending on the match 
between the test content and methodology and the desired program outcomes, this disconnect can be quite 
great. Further, given that most general education programs include choices of courses rather than a core of 
required courses, tests of general education content may be close to or far from the mark depending on the 
range of selection available and the actual course choice. 
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2. Student attitudes about the test It has been difficult to get accurate assessment of student learning in 
testing situations in which there are no individual outcomes for the student. Students see a big disconnect 
between a test they are required to take for institutional assessment purposes and those that advance their 
grades, admission to professional or graduate programs, or scholarship or achievement recognition. 

3. Difficulty in linking test results to improvements in the program. The feedback loop, using assessment 
to improve the program, is the primary justification for outcomes assessment. It is generally quite difficult to 
connect test results with specific programmatic changes, in part because the results are often more global. 
For example, even when scores of common sub-areas of general education, such as the social sciences, are 
available, relating the results to a specific course, e.g., economics, is difficult Going the next step to improving 
the learning outcomes for students who complete the specific course is even more difficult. 

Assessment models additionally rely on student surveys, alumni surveys, focus groups, and the like. These secondary 
measures can be helpful in assessing attitude, carryover of knowledge, and even perceptions of program rigor, but 
they do not assess actual learning outcomes. At the University of Northern Colorado, for example, in addition to a 
standardized test, the former assessment model included a course review process that assessed student opinion on 
whether the course met the generai education uiiieiia for the area (e.g., social science, arts and letters, composition, 
etc.) and whetherthe course syllabus reflected attention to the criteria and learning outcomes, but did not assess the 
actual learning. 

Additional factors can influence the effectiveness of general education assessment. First, general education courses 
can serve more than one academic program and, therefore, have competing assessment goals, which can create 
difficulty for faculty teaching the course as they try to assign priority to various student outcomes. For example, the 
major/discipline will have expectations for specific student learning outcomes, the general education program will 
have expectations for specific student learning outcomes, and other programs requiring the general education course 
(such as teacher education) will have expectations for specific student learning outcomes. 

The second issue is the one of accountability. Members of the governing body for general education, which at 
Northern Colorado is the general education Council, are usually elected, and the makeup of the body changes. That 
can lead to issues of continuity and changing commitment to assessing student outcomes. 



Faculty Involvement in General Education Assessment 

Many of us working in higher education would agree that, for a general education assessment program to be 
successful, it needs to be faculty driven and governed. Faculty at Northern Colorado are directly involved in the 
formation and implementation of the general education assessment program. However, faculty involvement in the 
assessment of the general education program has yet to generate widespread acceptance. Many faculty resist 
involvement because they believe that course-embedded assessment will result in an increase in their workload. Still 
others remain unenthusiastic because they feel that general education assessment is a passing fad and will not be 
sustained as council leadership and/or administrations change. 

In order to gain faculty acceptance of the new general education assessment model, a series of workshops was 
conducted during the fall semester of 2000. Faculty mem bers who have piloted the model and members of the general 
education Council who were familiar with the model led the workshops. The faculty who participated in the training 
agreed to use the model in their general education classes during the spring 2001 semester, after which these faculty 
will serve as mentors to other faculty members as they begin to implement the model in their respective classes. We 
suggest that the general education assessment model will gain widespread faculty acceptance because it was 
designed and implemented by faculty teaching in the general education program. In addition, course-embedded 
assessment provides the faculty with immediate feedback concerning student performance on general education 
learning outcomes, and the instructor can use that feedback to reorganize or restructure the course. Finally, the 
instructor can evaluate the assessment tasks used to measure student performance and make appropriate changes. 



The Role of Administration in General Education Assessment 

On most campuses, general education oversight and responsibility is assigned either to a dean or an academic affairs 
administrator, but the governance of the program is assigned to a faculty committee. As has been noted, the attitude 
of faculty about assessment has often been skeptical. If a general education assessment program is to lead to 
programmatic improvement resulting from assessment, faculty must, at minimum, be participants in assessment 
nlanning and administration, and ideally, they should lead it. That means that they need to have the support to explore 
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assessment options, propose models, and pilot and implement them. If programmatic change is to work, even when 
time constraints are critical, faculty involvement cannot be overlooked. 

At Northern Colorado, assessment planning and implementation occur at the college level, with a University 
Assessment Council setting standards and processes and facilitating information sharing. The Dean of Arts and 
Sciences has responsibility for general education and had three goals for the assessment program: that it would be 
a meaningful program leading to program improvement, that it would be developed and supported by the General 
Education Council, and that it would be manageable for faculty and administration. As a result, the dean can be an 
advocate for a faculty-developed model and a facilitator for its implementation. In essence, the assessment of general 
education at Northern Colorado has become a collaboration between faculty and administration, with faculty in a 
critical leadership role. 



Developing and Piloting Course-Embedded General Education Assessment 

During the 1 997-98 academic year, the UNC general education Council developed and implemented student learning 
outcomes for all skills and content areas of the general education program. At that time, the council had been using 
the ACT-CAAP test to measure student performance. The CAAP tests were used to compare the performance of a 
sample of UNC students with the performance of students nationally. However, after reviewing several years’ worth 
of CAAP test results, the council determined that the CAAP test was not the best instrument to measure general 
education performance expectations. 

In response, the council appointed an assessment subcommittee to examine various alternatives for assessing 
student outcomes. The subcommittee recommended to the council that assessment of student performance be both 
instructor- and course-specific. The members of the assessment subcommittee concluded that course-embedded 
assessment was the most effective way to measure student performance. In response to the subcommittee’s 
recommendations, the council directed a member of the assessment subcommittee to develop and pilot a course- 
embedded assessment model for the general education program. 

The faculty member selected for the pilot began by examining his course objectives and the student outcome 
expectations developed by the General Education Council. Rather than create new performance tasks to assess 
student learning, he aligned existing assignments with the student outcomes expectations. The idea was to make the 
pilot faculty friendly and to minimize the increased workload for faculty. In the initial pilot, the faculty member used 
a combination of exam questions and a capstone mapping project to measure student outcomes. For the course 
(World Geography, a freshman level class), the general education student learning outcomes were: Cl) the student 
will understand the framework, world view, or philosophical assumptions of the discipline; [2) the student will 
understand the methods and research skills used by the discipline; and [3) the student will understand the principal 
theories of the discipline. The capstone mapping project was used to assess outcomes 1 and 2, while a series of exam 
questions administered throughout the semester were used to assess outcome 3. [Note: in the second semester of 
piloting, the instructor dropped the exam questions as a measure of outcome 3 and substituted a requirement from 
the capstone mapping project.) 

One of the challenges to making the model successful was to keep the assessment processes and the reporting of 
the results as simple as possible. Using existing tasks and matching them to general education outcomes reduces 
the burden placed on faculty who adopt course-embedded assessment. To simplify the reporting process, a one-page 
form was developed to summarize the results of the assessment. In addition to basic course information, the instructor 
reports the student outcomes assessed and student performance levels associated with each outcome. The instructor 
also briefly explains the tasks used to measure each outcome [ie., exams, research projects/papers, presentations, 
essays, or class assignments) and the performance criteria used to evaluate the outcomes. 



Implementing the Course-Embedded Model 




Within the parameters discussed above, the implementation of the assessment model is the responsibility of the 
general education Council, but it is facilitated and promoted by the Dean of Arts and Sciences, with staff support. The 
new assessment model at the University of Northern Colorado is being implemented in stages for two reasons. First, 
the council is in the process of integrating the model with the course review process [adding the assessment of 
learning outcomes to the prior requirements of general education) and determining the guidelines for the range and 
frequency of assessment and reporting assessment results. Second, this is a faculty model, and faculty are training 
their peers to implement the model. 
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Currently, 20 faculty across the three skills and four content areas are implementing assessment in their general 
education courses. They will serve as training leaders in workshops to extend the program across the full range of 
general education courses over the following academic year. By phasing in the model beginning with interested 
volunteers, those least committed to the idea of assessment will, at the stage of full implementation, have a range of 
models from prior use available to them as well as good support from their peers. Finally, staged implementation 
allows for adjustments to address unforeseen issues as they occur and to have the benefit of that knowledge in the 
next stage. The goal of the general education Council is to reach full implementation through encouragement and 
persuasion, though the council has the power to require implementation and to remove courses from general 
education if they are not assessed. Members of the council believe that those who participate will see a direct benefit 
in their classrooms and become advocates of local control of assessment at the course level. 



General Education Assessment Integrated with Program Review 

In order to systematize and increase the impact of assessment on programs, assessment has been integrated into 
the program review process at the University of Northern Colorado. The state of Colorado requires a program review 
of all academic programs at least once every five years. Historically, this review had been focused on degree-granting 
programs. The decision to add general education to the program review process placed general education into a 
calendar of specific review procedures that established a program of activities and served to fix accountability for 
program planning and review and student outcomes assessment. 

Program review at Northern Colorado is focused on assisting programs with improving educational quality within the 
context of the principles identified by the university role and mission statement as well as the University Plan. The first 
step in the process is the identification of an external consultant who visits the campus prior to the development of 
a program self-study and whose report is responded to as part of the self-study. The program review process then 
moves to a program self-study that addresses a range of topics, including program background and characteristics, 
program goals and objectives, and undergraduate and graduate program performance and achievement through 
such means as analysis, goal setting, resource identification, faculty participation, and outcomes assessment. 
Following completion of the self-study, a Program Review Committee of faculty develops an initial report and 
recommendations. Following review and discussion with the Program Review Committee and college administration, 
the dean submits a final report and recommendations for the next five years. 

Beginning in 1998, student outcomes assessment data and the use of various measures for obtaining student 
feedback, including surveys and student focus groups, became an integral part of the departmental self-study 
process. The self-study must include a report on assessment data and on the programmatic changes over the past 
five years resulting from outcomes assessment. Additionally, the self-study may include recommendations for 
revision to the Student Outcomes Assessment Plan, which are then reviewed through college procedures. In addition, 
to maintain an ongoing focus on assessment, each program must provide annual updates on program review 
recommendations, including annual profiles of assessment activities and programmatic change. 

Prior to the inclusion of general education in the program review process, assessment was less systematic and the 
role and timing of assessment activities for faculty elected to the general education Council was not well developed. 
By placing the general education assessment process into the program review framework, the assessment process 
became part of an organized structure of institutional expectations regarding the assessment of student outcomes 
for program improvement. Requiring significant faculty involvement and the evaluation of an outside reviewer 
provided grassroots ownership to the assessment process that moved it away from a top-down analysis. At the same 
time, the formal recommendation process provided the university community and the administration consideration 
of resource issues for program improvement from academic planning and budgeting activities. 



Sandra Flake is Dean of the College of Arts and Sciences at University of Northern Colorado in Greeley. 

James Doerner is Associate Professor, Geography Department, at University of Northern Colorado in Greeley. 
Richard Trahan is Asssociate Dean of the College of Arts and Sciences at University of Northern Colorado in Greeley. 
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General Education Assessment: 
A Multilevel Approach at 
Northern Illinois University 



Virginia R. Cassidy 
David Changnon 
Ruth Gold 

Like many other institutions, Northern Illinois University has been wrestling with the assessment of its general 
education program for a decade. Following the North Central Association’s regional assessment seminars in 1991, 
the General Education Committee (GEC), a faculty committee responsible for the content and quality of the general 
education program, began talking about how to improve the evaluation of student learning in the program. A review 
process for individual courses in the general education program was already in place, but this review did not seem 
to capture the richness of the program in its entirety, nor did it address all the goals of the program. As a result of 
these early conversations, the GEC concluded that it was unable to develop better assessment tools unti I the mission, 
goals, and philosophy of the program were clearly understood. 

The committee’s next step was to embark on a multi-year self study beginning in January 1993. Various models of general 
education were explored; faculty and students were asked their perceptions of the goals for general education; the 
curriculum of and advisement for the program were examined; and a number of improvements were made: 

1. A general education mission statement was developed. 

2. The general education program goals were clarified and were restated in terms of student outcomes. 

3. A philosophical definition of general education that differentiates a course acceptable for general education 
credit from other courses in the curriculum was written. 

4. The guidelines for submission of a course for general education credit and for the review of existing courses 
were revised. 

5. A list of possible assessment tools at the course level, across courses, and program-wide was compiled. 

6. Recommendations were made regarding the structure of the general education program (e.g., course 
clustering, writing intensive courses, capstone experiences). 

This session will address how those improvements were implemented, particularly how the assessment of general 
education has evolved since the committee completed its review. In most cases, the impact of general education is 
difficult to assess because students select general education courses from a broad menu of available classes; they 
may take their course work at more than one institution; and they take the courses at different times throughout their 
baccalaureate experience. The GEC believes that a multilevel approach to general education assessment addresses 
some of the issues associated with the variations in course-taking patterns that result in individualized general 
education experiences. 

General education is an integral part of Northern Illinois University’s baccalaureate experience. Although students 
are free to select from a broad menu of general education courses to meet university requirements, these 
requirements are divided into core competency and distributive studies areas, which include the humanities and the 
arts, the sciences and mathematics, the social sciences, and interdisciplinary studies. The types of courses and the 
number of semester hours required in the core competency and distributive studies areas are prescribed. The total 
number of courses available for general education credit is not limited, but a rigorous evaluation of courses included 
in the general education program is conducted. 
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The GEC is charged with the responsibility for ensuring that the goals of the general education program are met. 
The committee discharges its responsibility by assessing the general education program as a whole as well as the 
constellation of courses that comprise it. The committee reviews proposals for new courses on an ongoing basis and 
reevaluates existing courses in the core competency and distributive studies areas on a rotating schedule over an 
eight-year cycle. 

Assessment initiatives at each level-course, program, or university-are often complex and are usually met with 
resistance from many faculty; therefore, success in implementing assessment activities must be linked to a positive 
outcome for those involved. The GEC has taken the approach that assessment activities should help faculty learn 
about how their courses fit into the general education program and generate innovative measures to examine whether 
students are developing basic skills. Because the university does not have a day designed for assessment of learning 
outcomes, it must be integrated into the structure of existing courses. Support for these assessment activities and 
the education of faculty about the value of assessment for the improvement of teaching and learning requires ongoing 
efforts. Modifying the academic atmosphere associated with assessment activities is paramount if the process, at 
each level, is to achieve higher standards. 

Northern Illinois University’s diverse student population brings other important assessment issues, such as pedagogy, 
multiculturalism, and honors requirements, to light. The GEC believes that faculty who teach courses in the program 
should describe the teaching methods and techniques used to address the different learning styles, values, and 
experiences of students and to improve information transfer to all students. Each course also must "promote a 
multicultural perspective” and demonstrate how it “addresses the university objective of increasing the understand- 
ing of diverse cultural heritages.” Assessing these course-related issues is often difficult because the terms 
multiculturalism and diversity can have different meanings to different individuals. Northern Illinois University is in the 
process of developing a comprehensive statement that will be used to assess whether these issues are being 
addressed. Uniform standards also need to be established in honors sections of general education courses so that 
students understand what will be expected of them when they take these courses. The GEC must rely on the expertise 
of others on campus to guide their actions in addressing these issues. 



Course-Level Assessment 

In its review of individual courses in the general education program, the GEC requires course objectives to be linked 
to the goals of the program, and the assessment of students’ performance in the course to provide evidence of 
progress toward goal attainment. To assist the committee in evaluating the appropriateness of courses submitted or 
resubmitted for the program, the GEC developed a new set of guidelines and materials for the faculty. The materials 
include examples of methods appropriate for the assessment of both course objectives and program goals. Faculty 
use these resources to prepare course proposals that clearly delineate the articulation of course objectives with 
program goals and the use of in-course assessment methods that will produce evidence of student learning 
outcomes. The committee also reviews the course syllabus to determine whether the general education goals 
associated with a course are being highlighted in the content and teaching/learning activities of the course. The 
emphasis on outcomes assessment has required the committee to become involved in outreach activities, and 
committee members serve as consultants to faculty members, assisting them in the preparation of their course 
proposals. In addition, the university has allocated resources to support faculty who are interested in developing new 
and innovative assessment methods for their general education courses. 

In an effort to help faculty in their submission of courses for inclusion in the general education program or in their 
periodic review documentation, many samples, examples, suggestions, and ideas for assessment methods have been 
made available. The most current resource tool is a set of instructions for developing course-level assessment. The 
dpcument provides step-by-step instructjpns and examples on matching course objectives to general education 
goals; identifying the cognitive functioning students use to achieve the objectives; defining types of assessment tools 
that evaluate across different levels of cognitive functioning; evaluating whether the existing evaluation tools are 
sufficient to provide evidence that the goals of general education are indeed being met; and describing how the 
assessment findings are used to improve the course. 



Across-Course Assessment 

To assess the general education program as a whole, the GEC designed a survey for faculty teaching courses in the 
program. The purpose of the survey is to determine how the learning objectives of the courses currently included in 
the program contribute to its goals. Rather than wait for a full eight-year review cycle of courses to occur, the 
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committee will use the survey information to construct a matrix plotting course objectives against general education 
goals. The matrix will assist in highlighting areas of strength [i.e., goals are more than adequately met) and weakness 
(i.e. T goals are partially or not met by an adequate number of courses) across the entire program. The committee will 
use the matrix in determining whether newly proposed or resubmitted courses would enhance the overall program 
and to identify where the program should allocate its resources in the future or solicit proposals to eliminate or 
minimize inadequacies. This across-course assessment is important because it will allow the committee to 
continuously improve the quality of the overall program, which can ultimately improve student learning outcomes and 
provide a more worthwhile baccalaureate experience. 



University-Level Assessment 

Over the last several years, four initiatives have been implemented to assess the outcomes of general education. As 
one component of the assessment of general education goals, the university initiated a junior-year writing project in 
1999. Departments preparing for program review or professional accreditation are invited to participate each year, 
and other programs can also elect to participate in the project. Students in junior-level courses in the programs are 
asked to produce a writing sample based on project criteria. These writing samples are read by faculty from the 
Department of English using a holistic grading rubric to assess defined aspects of writing competency. Individual 
feedback on performance is given to each student, and information on the student cohorts is sent to the participating 
programs to augment their assessment data. 

To date more than 1200 students have participated in this project. The results of the project show an overall 
satisfactory level of writing competency among the junior-level cohorts, but wide variance in competency among 
individual students. The project has resulted in the creation of a writing prompt applicable to students in all 
undergraduate programs, the development of junior-level writing criteria, and the recognition that the qualitative as 
well as the quantitative analysis of writing samples provides the richest assessment of students’ attainment of this 
core competency. Coordinating this ongoing project with the program review process is expected to give a university- 
wide picture of students’ writing ability over time. 

In fall 2000, two new initiatives to assist in assessing the attainment of general education goals were implemented. 
With new state funding, Northern Illinois University was able to offer faculty the opportunity to create or refine 
capstone experiences within their programs or to engage in training to develop and use student portfolios to provide 
evidence of learning outcomes. Stipends are available to faculty who are interested in developing a culminating 
experience within their degree program, or in thoroughly revising an existing capstone course. Many degree programs 
already have such experiences in the form of courses, team experiences, field placements, or supervised practica. 
Some experiences are very good, with assessments in place for evaluating the broad goals of the program. Other 
degree programs either have not yet developed capstone courses, or have not clearly associated their capstone 
course objectives with general education or program goals. 

Portfolio training will enable faculty to design student portfolios in a variety of approaches that include web-based, 
CD-ROM, or paper formats. The portfolios can be used to document learning outcomes in the degree program. Both 
initiatives must be designed to provide evidence that the broad goals of the baccalaureate experience, particularly 
general education objectives, such as writing, critical thinking, quantitative reasoning, an appreciation for 
multiculturalism, or the use of technology, are being met. Samples of these indicators will be collected as indirect 
evidence of the extent to which students have achieved the goals of general education by their senior year. 

In spring 2001 the university will also undertake the assessment of critical thinking skills among sophomore-level 
students. Approximately 500 students enrolled in programs in four colleges will complete the ACT CAAP Critical 
Thinking Test. This project will provide the departments and the GEC with data on critical thinking skills as students 
complete a significant portion of the general education program and begin coursework within the major. Critical 
thinking skills will be reassessed when these cohorts of students are enrolled in their final semester of study. 



Virginia R. Cassidy is Assistant Provost, Academic Planning, at Northern Illinois University in DeKalb. 
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Ruth Gold is Coordinator for Assessment Services, Office of the Provost, at Northern Illinois University in DeKalb. 
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Planning, Assessment and Budgeting: 
A Web-Based Solution 



Joanna K. Michelich 
Carlos Cartagena 
Janet Martinez-Bernal 



Introduction 

“The best thing for disturbances of the spirit,” replied Merlin . . . “is to learn. That is the only thing that never fails.” 

-T. H. White, The Sword in the Stone 

Cochise College is a comprehensive community college located in southeastern Arizona bordering both New Mexico 
and Sonora, Mexico. The largest rural community college district in Arizona, Cochise College serves more than 1 0,000 
students annually through its multiple sites, which include two comprehensive campuses, three education centers, 
a prison education program, a rapidly growing online campus, and numerous community education locations 
throughout Cochise County. Due to its unique location, Cochise College serves a tremendously diverse community, 
which includes a significant Hispanic student population at its Douglas Campus as well as highly skilled technical 
personnel associated with Ft. Huachuca, the Army’s training headquarters for both Military Intelligence and Army 
Signal Command. 

In the mid-1990s, Cochise College defined its institutional effectiveness model as one that linked and integrated its 
planning, budgeting, and assessment processes. Like many other institutions, however, Cochise College experienced 
a numberof false starts, incomplete paths, and disappointing results as it worked toward implementing its integrated 
model. Faculty and staff throughout the institution demonstrated an uneven understanding of assessment; budget 
managers had difficulty linking assessment data to budgetary requests; and the institution lacked a comprehensive 
strategic plan. Since that time, significant progress has been made under the direction of new presidential leadership 
in each of the three areas [planning, budgeting, and assessment]. However, it has been the application of technology 
that has enhanced the institution’s ability to design a way of operationalizing the integrated model at the user level. 

Cochise College’s experience with developing a web-based tool for linking planning, assessment, and budgeting may 
provide other institutions with one means that has enabled Cochise College to overcome the disconnects between 
and among these three critical institutional processes. 



Background and Context 

Cochise College’s history with integrating and linking planning, assessment, and budgeting might best be traced to 
1996 as a result of a comprehensive NCA visit. While the visiting team accurately noted “the college expects to 
integrate institutional effectiveness effort, the results of assessment effort..., and other elements of institutional 
planning including environmental scanning and operational (budget] planning, into a single integrated system,” it 
also noted that the college was “still struggling to define clearly the essential components." 




Stating that the college met Criterion Four in “more than a minimal sense,” the team also pointed out the most serious 
challenge facing the college as it sought to integrate assessment, planning, and resource allocation was the “lack 
of attention paid to assessment of student outcomes.” In addition, the team cited the lack of “clear environmental 
scanning that gives direction to educational planning or budgeting," and indicated that program reviews “do not seem 
to make impacts on institutional budgeting or decisionmaking,” and “levels of understanding among constituent 
groups about the college direction vary considerably.” 1 
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Immediately following the team’s visit in spring 1996, the college was faced with major challenges on several fronts 
(changes in leadership and adaptation to state mandates being the most significant). The curricular and other 
changes occurring between 1996 and 1998 moved the college toward a greater awareness of the importance of 
student learning outcomes and began a much-needed updating of technology in support of both instruction and 
institutional infrastructure. However, it was not until summer 1998, when the current president was officially 
confirmed in her position, that the issues cited by the visiting team could be directly addressed. 



Statement of the Problem 

The problem was to link planning, budgeting, and use of data from assessment of student learning into a unified decision- 
making system, for the primary benefit of students, in support of the evolution of the college into a learning community. 



College Response: Assessment 

In fall 1998, an institution-wide Assessment Committee undertook a background study, which included information 
about NCA assessment expectations, a scan of existing data within the institution, and consultation with other, sister 
institutions in Arizona that had faced similar circumstances. To increase consistency of reporting, the college has used 
existing technology, in three different “editions” so far, with continued improvements in the works. 

□ First Edition 

In fall 1 998, it became evident that data were being collected, and in many cases being used effectively within units 
of the college; however, methods and use varied widely. There was little consistency over time or across the 
organization. Therefore, in spring 1 999, Assessment Committee members, working with the Director of Planning 
and Research, designed a template in Access based on James Nichols’ model for reporting data relevant to 
institutional effectiveness that could be applicable institution-wide. 2 

During spring and fall 1999, representatives of 30 different college units used this template to state their department 
missions, which aspect of the college mission theirs most closely supported, what outcomes they hoped to achieve 
in support of their mission, and how they would measure their success in doing so. Participants ranged from aviation 
and English faculty to maintenance staff and student services professionals. They were coached through the 
template individually or in small groups, usually by the chair of the Assessment Committee. Each approached the 
task from a different set of experiences and a different perspective, but all struggled with the format and the 
vocabulary. Most voiced skepticism that the statements they produced would ever be of much practical use. 

Despite the skepticism, the statements produced and the thinking engendered by the process of using this first 
edition of the template served as an introduction to the kinds of linkages that were soon going to be required. In 
fact, even on this rudimentary scale, participants produced department mission and outcomes statements that 
addressed 17 out the 25 segments of the college mission to which their goals could be connected. 

However, at this stage, the use of the template was perceived as serving only the purposes of the Assessment 
Committee, which were not seen as directly or meaningfully related to the routine processes of the college. 

□ Second Edition 

By the end of the academic year 1999-2000, attention turned to reporting results for the current year and beginning 
the planning process for 2000-2001. A second edition of the template, updated in response to feedback by the 
previous users, was provisionally posted on the web for experimental use. The template, as then designed, could 
be used in two ways. Those managers who had used its format to present their plan for the year could simply “roll 
it over” into a report, add their results and their tentative plan for the next year, and be finished. Those who had 
used a different format for their plan were asked to use this format for their reports only, not to restate their original 
plan. Instructional managers spent an afternoon in a workshop seeing examples and drafting their own practice 
documents in small groups before using the posted template. 

This experiment revealed several problems. First, some administrators, perceiving the template as serving only the 
purposes of assessment, backed away from this format and did not require its use, or required additional parallel 
reports from those reporting to them. Second, it became evident that budget managers varied widely in their ability 
to use a new web-based application, even a relatively uncomplicated one. Technological anxiety, added to the usual 
end-of-year stress, hampered use. Finally, flaws (especially ambiguities) in the template itself became obvious when 
it was presented to this larger group of users who could not be coached individually. 
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The most positive consequence of this experiment was that users emphatically communicated what was not working. 
The problematic areas of content and process were apparent from a cursory study of the results; with these new findings 
in hand, a team of assessment and web design people went back to the drawing board (story board) to produce a fully 
integrated system for use in the 2000-2001 year. This became the “third edition,” the basis of our current process. 

College Response: Planning 

It should also be noted that during calendar year 2000, an institutional strategic planning process began; a 
comprehensive environmental scan was completed, a preliminary vision statement drafted, planning assumptions 
identified, community town halls conducted, and several planning retreats held. These efforts culminated in the senior 
administration’s articulation of eleven strategic college-wide goals designed to guide the institution over the next five 
years. Thus, the prior concern about the lack of a comprehensive strategic plan had also been mitigated. 

College Response: Budgeting 

Perhaps the weakest link in the integrated model, the budgeting process, heretofore had tended to operate outside 
the planning and assessment processes. Budget requests were forwarded to senior administrators, at times 
accompanied by assessment data, but more often as simple requests for additional resources either to enhance 
existing programs or to support new initiatives. Senior administration was left to infer the value of resource requests 
based either on anecdotal information or on perceived value based on minimal factual information. In essence, 
although the relationship of requested resources to college mission and values was considered, references to 
assessment data and results were not integral to the decisionmaking. Effective with the 2001 -02 budget development 
cycle, a commitment was made that all budget requests for increased or new funding would: 

o be required to relate to one of the eleven college-wide goals, and 

o be documented and substantiated based on assessment data (direct measures, indirect measures, or general 



Web-Based Integrated Tool 

Having in the hand the requirements of the planning, assessment, and budgeting process, we explored different 
venues in order to automate it. It became evident that adopting Internet standards into our project would optimize 
its value. We decided to conceptually design a system that would be user-friendly, dynamic, and database-driven. 

With the project requirements identified, the web development team went into action. The team decided to develop 
the assessment web site using a combination of Hypertext Markup Language (HTML), ColdFusion Markup Language, 
(CFML), and an Access database. The project front-end (user interface) was designed following established HTML 
standards using FrontPage as the web design tool. On the back end, powerful CFML code connected to a database 
running on a ColdFusion server provided the capability to add, edit, and delete data as users deem necessary. In 
addition, it allowed the capability to filter users’ access based on a security login process. This feature restricted the 
number of records that a user could view depending on the user’s access level. 

“ColdFusion is a rapid application development tool that enables the rapid creation of interactive, dynamic, and 
information-rich Web sites.” 3 CFML is very similar to HTML and extends its functionality. The web development team 
found that, although this web site would be a very complex project, using Internet standards would minimize the 
impact associated with the implementation of a new planning, assessment, and budgeting system. The main benefits 
of these standards are the capability for a user to access the site from any location where an Internet connection is 
available and the institution’s capability to keep very valuable planning, assessment, and budgeting data in a reliable 
and powerful database repository. 

The concept (storyboard) for this project called fora user to enter a web address into the browser and access a login 
web page. After the user entered an ID and password, the system would load a web page containing the college’s 
mission and vision statements and a text area for the user to enter the departmental mission statement. The next step 
was for the user to select the instructional unit, academic year, and campus supported by this record. Each one of 
these fields was based on the select list features afforded by HTML. Then the user would select one of eleven strategic 
goals that were developed by the senior administrative team. By doing so, the record would be directly associated 
with an official college objective that was part of the institution’s long-term strategy. 



indicators). 



ERIC 




286 /A Collection of Papers on Self-Study and Institutional Improvement , 2001 

The next steps would require users to enter supporting data, briefly explaining the following: 

1. Department/unit goal: State what you hope to accompHsh-your intepded outcome or objective within your 
department. Identify up to three department goals. Please enter one goal per record. 

2. Activities/strategies to achieve the goal/outcome: Identify up to three for each department goal. 

3. Rationale: State background information and assessment results that support the proposed goal/outcome. 
Please limit your rationale to a maximum often lines. 

4. Assessment of activity: How will you know the activity made a difference? 



At this point, the planning and assessment portion of the process was completed, and the user was afforded the 
opportunity to input funding data if there was a need to request new resources in order to support the planning or 
assessment activities. This web form was automatically loaded to the user’s browser the moment that the assessment 
block was completed, and the data were saved in the database. The web budgeting form contained areas for increases 
in operational funds, new personnel requisitions, and capital requests. The system wm ild keen a subtotal tally for each 
one of these categories as well as total dollars requested for the record. 

Users’ feedback indicated that the college’s planning, assessment, and budgeting web site met its original goals. They 
indicated that the site was user-friendly, was easy to navigate, had clear instructions, was fast to download, and was 
a welcome improvement over past assessment practices. The only problem some users experienced was forgetting 
their password, but resetting it was a very simple process for the site administrator. We plan on implementing some 
improvements for the 2002-2003 budget cycle in order to make the site even more user-friendly. 

If you are interested in viewing and testing this web site, please point your browsers to http://www.cochise.org/assess 
and use “guest@c 0 chise.cc.a 2 .us” (without the quotation marks) as the login and “guest” (again without the 
quotation marks) as the password. We would welcome your very valuable feedback as well. 



Results 

We have been pleased with the results of our pilot web-based tool in its first year. Positive results have included: 

1 . Budget managers have indicated that the process was user-friendly and efficient. 

2. All units that had not previously written a departmental mission statement now had to do so. Unit managers 
have self-reported that the process resulted in good internal dialogue regarding the purpose, mission, and 
goals with faculty and staff in their respective areas of responsibility. 

3. Assessment became a familiar concept for some areas of the institution not directly involved in the learning/ 
teaching process; several areas conducted fairly extensive assessments of their areas for the first time. 

4. Administrators have been able to review assessment data with each budget request and have been able to 
rely on data rather than subjective assessment of the value of each request. 

5. There has been an increase in budget managers’ understanding of the relationship between and among 
planning, assessment, and budgeting. 



Remaining Challenges and Future Use 

Our results indicate that Cochise College still has additional work ahead to fully integrate planning, assessment, and 
budgeting, including: 

1 . Some tweaking needs to occur to make the web-based process even more user-friendly. 

2. We have identified several areas of the institution that clearly need training about the purpose of departmental 
mission statements. 

We found that differentiating between mission and goal, and between an outcome and an activity or means 
of achieving it, was difficult for novice users. Additional training will be necessary to assist managers in this 
area as well. 
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4. There is a definite need to strengthen understanding about assessment measures. Some managers cited 
unsubstantiated opinion as evidence; other managers stated “none available” when simple inquiry or search 
would have easily yielded facts to support their requests. 

5. Having overcome the technological barrier, the next steps will be to develop standards for budget preparers 
and criteria for budget decisionmakers. Learning will take place through the natural consequences of reward 
in the funding of some decision packages and of rejection in the case of others. 

6. Finally, the institution itself will need to demonstrate by way of budget decisionsyetto be made that the process 
was utilized in allocation decisions. Because of the institution’s budget calendar, those final decisions will not 
be made for several more months. 



Conclusion 

Using today’s technology to implement an important institutional imperative has led to positive results. This 
presentation will include information on one institution’s experience with designing and implementing a web-based 
tool to implement an integrated planning, assessment, and budgeting process. Progress on general understanding 
of the complex interrelationships and shared responsibilities for these three integrated processes is documented and 
will be reported. Managers’ experiences using the tool will also be discussed, as will the next steps in its evolution. 
The presentation will include a live demonstration, followed by practice in small groups utilizing the elements 
contained in the model. Copies of the elements contained in the tool will be made available to attendees. 
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The Self-Study's Role in 
Creating or Increasing Momentum for 
Significant Institutional Change 



Karen i. Haibersieben 
Robert Ferguson 

Given the accelerating pace of change in higher education today, schools can no longer afford to put all other pressing 
issues to one side while they focus on re-accreditation. At small or medium sized schools in particular, leaders should 
craft a reaccreditation process that allows for an institutional review of honesty, depth, and integrity, but one that 
leaves human resources available for the concomitant work of constructing and carrying out institutionally important 
change. Buena Vista University (BV] undertook several significant, wide-ranging change initiatives during the two 
years it was also involved in its self-study process. To a significant extent, the analysis of issues generated by the self- 
study either created or strengthened the will to change in several key areas. Strategically, the self-study process and 
report provided timely and well-documented support for an important range of institutional improvements on 
campus: 

0 Curriculum. The focused analysis of the university’s mission statement necessitated by the self-study caused 
the Chief Academic Officer to redraft the timetable for a complete overhaul of the curriculum, moving the 
reform movement forward by two years. The need to demonstrate that the curriculum supports the overall 
mission of the university provided a rallying cry for the faculty as they successfully undertook an accelerated 
two-year curriculum reform. 

0 Personnel policies. As the self-study accentuated the gap between the new reality of BV at the end of 2000 
as opposed to 1990, the faculty's willingness to participate in a major revision of its personnel policies was 
strengthened. The document they produced effectively aligns BV's values as a teaching institution with its 
hiring, development, and reward system. 

0 Distance learning. Pressing needs for focused attention to the distance learning program emerged as the 
self-study pulled together, producing a significant reorganization of BV's 1 7-site Centers program as well as 
the development of key policies and procedures that clearly indicated faculty control of the academic program 
throughout the system. 

0 Technology. The chronicle of decade-long, leading-edge technological innovations depicted in the self- 
study gave us the courage to stay the course, becoming the nation’sfirst completely wireless, universal-access 
campus. 

All of these important improvements occurred at the same time the self-study was in process. We believe the reason 
so much significant work was possible-the self-study plus the reform work in addition to an already heavy all-staff 
workload-was because the university organized the self-study in a way that involved the entire campus but did not 
divert so much time and focus that work on other key initiatives became burdensome. The President's Council also 
announced their intention to use the self-study to establish key initiatives for the university, imbuing the entire 
endeavor with heightened relevance for the campus. BV organized the self-study using two-member team leadership 
for each criterion: a faculty member and a middle- or upper-level administrator. Each criterion team was charged: 



o to use their portion of self-study to identify areas in need of immediate attention; 

o to build connections between the findings of the self-study and major reforms that might already be underway 
on campus; 
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o to generate research questions on relevant topics. Survey questions were gathered from all five criteria groups 

and organized so that relevant questions were sent to each stakeholder group only once. Results were fed 
back to the entire campus, preparatory to follow-up focus group meetings. 

The entire campus community could access drafts of the self-study on the Web; several availed themselves of the 
opportunity to submit amendments and suggestions. The campus participated at strategic points throughout the 
process, as they attested to the NCA team while the latter was on campus. Just as importantly, the Buena Vista process 
allowed the self-study both to be immediately useful in agenda setting and also not to soak up more human resources 
than a small, change-oriented campus could afford. 



Karen I. Halbers/eben is Vice President of Academic Affairs and Dean of Faculty at Buena Vista University in Storm Lake, 
Iowa. 

Robert Ferguson is Professor of Psychology at Buena Vista University in Storm Lake, Iowa. 
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Self-Study to Planning: 
Institutional Improvement by Design 



John Mosbo 
S. P. Buchanan 
Elaine McNiece 
Jonathan Glenn 



Contexts 

In a state with a population of 2.5 million, the University of Central Arkansas fUCA) is one of nine publicly funded 
institutions of higher learning at the baccalaureate level and above; in addition, the state supports twenty-two 
two-year institutions. UCA is the fourth largest in number of enrolling students and the third largest producer of 
student semester credit hours of the nine. It faces increasing competition for resources and students. The university 
has responded by more sharply defining its niche within the state community of higher education. 

In this context, UCA began in 1997 to prepare for a self-study that would culminate in a site visit in the spring of 2000. 
This preparation involved the principle of universality-the self-study was global in its scope and depended on a com- 
prehensive set of department-unit-process self-studies-and decisions about self-study design with that principle in mind. 

The principles and design of the UCA self-study required a good deal of groundwork, beginning with the review and 
rearticulation of the university’s statement of mission and purposes. More directly, the Steering Committee spent 
months on the formulation of key questions, on matters of schedule, on the formation of task forces-on, that is to 
say, the machinery of self-study. That machinery in place, the self-study director and co-director began a series of 
presentations with the aim of making sure that the campus was fully aware of the process, of its organization, of the 
questions driving it, and of the stakes. They wanted to reach-directly-every employee of the university or as near 
to that as was practicable. In a period of just over three months, they accomplished their goal, with presentations to 
thirty-six on-campus groups large and small (thirty of those in less than a month), including all academic departments, 
non-departmental academic units, administrative units, financial and physical plant units, and so forth. These 
presentations were supported by stories in the on-campus newsletter and in the student newspaper and by the 
creation of a web site making available information about UCA’s self-study and providing links to further information 
about accreditation in general. 



Mission and Roles 

This groundwork and the subsequent products of the self-study process precipitated dramatic changes at UCA, with 
two major interdependent outcomes of the process and its products. Preparation for the self-study and the self- 
studies themselves made obvious what many had sensed already-that UCA’s mission had gone out of focus and that 
its vision was blurred. Rearticulation of the university’s mission and the consequent sharper focus on institutional 
vision served to differentiate between the institution’s primary mission as a traditional undergraduate, residential, 
high quality institution and secondary roles driven by the flux of external expectation. Increased clarity of mission and 
the process of self-study led as well to a major restructuring of reporting lines within the academic division, allowing 
the provost to focus more clearly on the university’s primary mission and, with the associate provosts and deans, to 
provide more effective leadership in both “main-channel” and "off-channel” institutional roles. 

Main-Channel Change 

Most dramatically, the process and its products has led to a reexamination of academic programs in light of 
mstitutiona! vision and statements of mission and purposes. UCA’s rearticulation of its mission and purposes 
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prompted a conscious clarification and development of institutional vision and a subsequent academic positioning 
initiative. Undertaken partly in conjunction with and partly as a result of the self-study process, the academic 
positioning initiative was (and is) an exercise in alignment with mission and resources. 

Academic positioning is about asking and responding to questions-for deans, chairs, and faculty-about clientele 
(who are our students? who should they be? who are our other constituents?), about personnel (who is teaching our 
students? who should be? have we effected the optimum match of clientele and personnel?), about resources (are 
our resources being used in the right places, given our clientele and our personnel? how can they be better allocated 
to fulfill our place in the university's mission?), and about future directions (where do the answers to our questions 
lead us?). The initiative has thus far had mixed success. Where deans and chairs have recognized the seriousness 
of these questions and considered how this process can improve (and sometimes change) what they do, academic 
positioning has been successful: in some cases it has affirmed existing roles and in others led to substantial change. 
Where deans and chairs have had difficulty shifting from the growth mode of the 1980s and 1990s to the current 
reallocation mode, academic positioning has been less successful and at times painful, leaving these parts of the 
university in danger of stagnation, with no renewal in sight. 



Off the Beaten Path 

Outside-and sometimes in conflict with-UCA's primary mission, certain market-driven secondary missions exist: to 
provide applied graduate programs for the region, to deliver programs through distance education, and to meet needs 
for workforce development and nontraditional, less-well-prepared students of UCA’s immediate community. Though 
institutional mission and priorities drive normal planning and resource allocations, external expectations not 
accompanied by additional resources require new approaches to planning and management. The university has, 
perforce, developed innovative and entrepreneurial approaches to meet these secondary expectations without 
redirecting scarce resources from the primary mission. Two new academic units have been created; one has been 
restructured; and business plans have been developed to allow innovative programming in areas outside the primary 
mission of the institution. Balancing entrepreneurship with institutional priorities has been achieved by requiring that 
these new initiatives be self-supporting. New initiatives include: 

0 Creation of the new Graduate School of Management, Leadership, and Administration (GSMLA), a separate 
organizational entity to house the Master of Business Administration, graduate programs in educational 
administration, and future interdisciplinary offerings in management, leadership, and administration. 

0 Restructuring of the Division of Continuing Education and development of a business plan for its delivery of 
distance education. In support of one of the division’s off-channel responsibilities, the Director of Continuing 
Education developed a distance-education business plan to assume all costs associated with distance 
education delivery for UCA. 

0 Creation of a new two-year curriculum and student support mechanisms-a University College-within UCA. 
University College serves as a base for baccalaureate programs at UCA for less-well-prepared students, 
developing support services and curricula to meet student needs; offers students lower-division courses 
leading to an associate degree in general studies, or other associate degrees offered by the university; provides 
underprepared students with opportunities to achieve entry-level skills; and works with Continuing Education 
to improve employability and enhance career skills. 

Both the process and the products of UCA’s self-study have proven useful to the university, partly by design but also 
partly, if truth be told, by serendipity. The processes and products of self-study have provided response mechanisms 
and created opportunities for UCA. 



John A. Mosbo is Provost at the University of Central Arkansas in Conway. 
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Elaine McNiece is Associate Provost and Dean of the Graduate School at the University of Central Arkansas in Conway. 
Johathan Glenn is Director of Academic Planning and Assessment at the University of Central Arkansas in Conway. 
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Self-Study and Strategic Planning: 
A Critical Combination 



Clay Goodman 
Rene Willekens 



Abstract 

Estrella Mountain Community College began its first self-study in the fall of 1995 and received initial accreditation 
in the spring of 1 997. The challenges identified in this initial self-study served as the foundation for all subsequent 
strategic planning activities and related plans. The lessons learned are helping to guide Estrella Mountain’s current 
self-study (spring 2002). New challenges will be incorporated into an already dynamic strategic planning process that 
will take Estrella Mountain Community College into the future. 



Introduction 

Estrella Mountain Community College (Estrella Mountain) is the tenth and newest of the Maricopa Community 
Colleges, which serve Maricopa County, including the greater metropolitan area of Phoenix, Arizona. All Maricopa 
colleges are overseen by a Chancellor and Governing Board and are united by a common mission statement. Each 
college is separately accredited by NCA with a mission reflecting its unique community demographics and service 
area concerns. 

Estrella Mountain was the last of three new community college education centers approved for construction by 
Maricopa County voters in a 1984 bond election. Estrella Mountain began as an educational center, sharing 
accreditation with Glendale Community College. Plans called for the college to seek separate accreditation once it 
reached a suitable size. The college began offering evening classes through local high schools in 1990 and opened 
the first of four planned phases of the permanent campus in 1992. The campus began expansion of Phase II in 1998. 

As a result of its rapid growth, Estrella Mountain Community College Center sought initial accreditation from the 
Commission on Institutions of Higher Education of the North Central Association of Colleges and Schools. An 
institutional self-study report was submitted to NCA and the Evaluation Team in 1996. As a result of the NCA 
Evaluation Team visit and recommendations from the Review Committee, Estrella Mountain Community College was 
granted initial accreditation on April 7, 1997. 



Demographics 

Estrella Mountain Community College serves a very diverse community of approximately 200,000 residents. The area 
is 55 percent white (not Hispanic), 37 percent Hispanic, 5 percent African American, 1 percent Asian, 1 percent 
American Indian, and 1 percent other. The servicearea includes a community with the third highest per capita income 
as well as five other communities with household poverty rates of greater than 20 percent. 

Student demographics are as follows: 

o 61 percent of Estrella Mountain students are female. 

o More than 85 percent of Estrella Mountain students attend part-time (fewer than 12 credit hours), 
o 88 percent of full-time students are under age 25; 69 percent are 1 9 or younger. 
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o More than 31 percent of Estrella Mountain students are Hispanic. This represents the second highest 
percentage in the Maricopa Community College District. 

o More than 40 percent of full-time student equivalent (FTSE) is generated by students receiving financial aid. 

o Approximately 1 in 10 Estrella Mountain credit students is 50 years or older [spring semesters). This is the 
second highest percentage of 50+ year olds in Maricopa. 



Evolution of Planning at Estrella Mountain 

□ Historical Stages of Planning 

Since its inception in 1988, strategic planning has been an integral and ongoing part of campus life at Estrella 
Mountain Community College. It has gone through several very distinct stages. The first [1 988-1992) set the stage 
for the initial development of Estrella Mountain and centered on a set of six planning directions: 

o Collaborative strategic planning 

o Educational responsibility to West Valley communities 

o Comprehensive instructional programs and flexible approaches to instructional delivery 
o Partnerships and collaboration 

o Integration of information technologies across the curriculum 
o Strong linkages with public schools. 

The second stage, Insuring Institutional Effectiveness (1993-1995), focused on development of the organization, 
institutional vision, values, mission statement, and purposes. This second phase built on past planning activities 
and helped prepare Estrella Mountain Community College (Center) for its initial self-study. 

The third stage integrated the Institutional Challenges identified from the initial self-study into Estrella Mountain's 
Strategic Directions. These challenges were incorporated into the College Strategic Plan. The College's Leadership 
Council assumed the responsibility for shepherding the continued development of the institution's strategic and 
other plans. The Strategic Plan is intended to guide all planning and resource allocation processes including 
divisional plans. 

□ Focus for 2000-2001 : Divisional Planning 

Each academic and administrative division at Estrella Mountain is required to submit a divisional plan that includes 
linkages to the Strategic Plan. The Leadership Council developed a set of planning and process guidelines for use 
by each of the divisions that included common required components as well as recommendations for optional 
elements of the plans. The components include items such as divisional mission, objectives Oinked to the strategic 
plan), summary of resources, and a continuous improvement plan. The completed divisional plans will be used as 
a basis for divisional budget requests and campus budget decisions. 

□ Participatory and Continuous Process 

Estrella Mountain's planning processes are highly participatory. All employees are given the opportunity to 
participate and provide input into the various stages of the planning process. It is not uncommon to obtain a 
participation rate of 80 percent or more of employees in planning activities held throughout the year. Some 
examples include all-employee feedback sessions on Core Values and the most recent update of the Strategic Plan. 

Strategic planning is also a continuous process. The College's Strategic Plan is continuously monitored and 
updated by the College Leadership Council. Council membership includes all divisional chairs and directors and 
employee group representatives. 



Linkages Between the Self-Study Process and Strategic Planning 




The self-study process is often overlooked as a strategic planning tool. A self-study can provide feedback to improve 
the planning process as well as update the College Strategic Plan through the identification of new or expanded 
strategic issues. 
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The self-study process has similarities to a common planning tool called a SWOT analysis. SWOT is an acronym for 
strengths, weaknesses, opportunities, and threats. Strengths and weakness are internal to the organization, while 
opportunities and threats are external and represent the environment that the organization operates in. The tool is 
designed to help organizations identify strategic issues that should be addressed by a planning process. 

Virtually every self-study will identify a number of strengths and opportunities for improvement [similar to strengths 
and weaknesses in a SWOT analysis). The strengths and opportunitiesfor improvement constitute an internal analysis 
of the institution. A well-designed self-study will evaluate these strengths and opportunities in the context of 
anticipated trends in the environment Qe., legislative funding changes, demographic shifts, distance education). 
These trends are similar to opportunities and threats in a SWOT analysis. Challenges and recommendations that come 
out of this process should be incorporated into a college’s strategic plan. 

The authors contend that NCA’s five Criteria for Accreditation are also closely linked to the strategic planning process. 
The table below demonstrates linkages to the strategic planning process as well the potential impact on the process. 



NCA Criteria 


Link to 

Strategic Planning 


Potential Impact 
on Strategic Planning 


Criterion One: 

The institution has clear and pub- 
licly stated purposes consistentwith 
its mission and appropriate to an 
institution of higher education. 


Mission, 

Purposes [Goals) 
Strategy 


Serves as a thorough review, and may 
result in changes to mission and pur- 
poses (goals). 

In extreme cases, changes to mission 
and purposes may cause the institu- 
tions to reevaluate the programs and 
services offered to its publics. 


Criterion Two: 

The institution has effectively 
organized the human, financial, and 
physical resources necessary to 
accomplish its purposes. 


SWOT 

Budget Planning 

Decision Making Processes 

Strategy 


Identifies the institution’s strengths and 
weaknesses related to all forms of hu- 
man, physical, and financial resources. 

Provides an evaluation of the resource 
allocation process that may result in 
changes to budgeting and decision- 
making systems. 


Criterion Three: 

The institution is accomplishing its 
educational and other purposes. 


SWOT 

Budget Planning 

Institutional Effectiveness 
and Assessment 


Serves as an internal scan that can 
identify the institution’s strengths and 
weaknesses related to achievement of 
the College’s Mission Purposes. 

Identifies program and services that 
may need special attention in the plan- 
ning process and require additional 
resource investment. 


Criterion Four: 

The institution can continue to 
accomplish its purposes and 
strengthen its educational 
effectiveness. 


SWOT 

Budget Planning 
Decision Making Process 
Strategy 

Institutional Effectiveness 
and Assessment 


Identifies strategic issues that may 
challenge the institution over time. 

Provides an evaluation of the planning, 
budgeting, and decision-making pro- 
cess that can result in improvements to 
program and service delivery. 


Criterion Five 

The institution demonstrates 
integrity in its practices and 
relationships. 


SWOT 

Values 


Serves as a check on the institution’s 
integrity and values system, may result 
in changes to an institution’s values 
and/or mission. 
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□ How EMCC Used Self-Study to Foster Strategic Change 

The first self-study conducted by Estrella Mountain had a significant impact on the College’s strategic planning 
process. The College’s 1996 Institutional Self-Study resulted in a set of six Institutional Challenges and 36 self- 
identified recommendations for institutional improvement. These challenge areas and recommendations were 
communicated to all college stakeholders. As soon as the self-study was complete, it became apparent the 
challenges and recommendations needed to be linked to the strategic planning process. The six Institutional 
Challenge Areas were converted to College Strategic Directions. Recommendations were condensed into 
institutional priorities linked to the larger Strategic Directions. These Strategic Directions and Institutional Priorities 
are intended to guide all planning and budgeting processes. Below is a table that shows the relationships between 
the College’s original challenge areas and the resulting Strategic Directions. (Please note: Institutional priorities 
are not represented here due to space limitations.] 



1996 Self-Study Challenges Areas 


Current EMCC Strategic Directions 


Institutional Planning 


Planning and Charting Our Future 

Estrella Mountain must fully implement a system of planning and 
assess progress toward its mission. 


Growth and Development 


Growing and Expanding 

Estrella Mountain must be proactive in meeting the needs of a 
growing West Valley population 


Community Involvement 


Creating Partnerships 

Estrella Mountain must continue to engage in partnership activities 
that advance the mission of the college. 


Organizational Governance 


Investing in People 

Estrella Mountain must continue to develop and invest in systems 
that support becoming a quality-driven institution. 


Strategies for Information Technologies 


Integrating Information Technologies 

Estrella Mountain must continue to invest in technologies to support 
teaching and learning the development of new delivery formats. 
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□ Challenges Incorporated into College’s Strategic Plan 

The primary goal of the initial self-study was continuous improvement through the identification of institutional 
strengths and opportunities for improvement. A candid appraisal of strengths and challenges provided the 
framework for the College to build on its strengths and address its challenges. The challenges and recommen- 
dations for institutional improvement were a natural fit with a strategic planning process. Challenges identified in 
the self-study were organized into six major areas that form the heart of Estrella Mountain’s most recent Strategic 
Plan: 

o Planning and charting our future. Estrella Mountain must fully implement a system of planning and 
assess progress toward its mission. 

o Growing and expanding. Estrella Mountain must be proactive in meeting the needs of a growing West 
Valley population. 

o Creating partnerships. Estrelia Mountain must continue to engage in partnership activities that 
advance the mission of the College. 

o investing in people. Estrella Mountain must continue to orgariize and develop its human resources to 
meet the changing organizational needs. 

o Creating a climate for success. Estrella Mountain must continue to develop and invest in systems that 
support becoming a quality-driven institution. 

o Integrating information technologies. Estrella Mountain must continue to invest in technologies to 
support teaching and learning and the development of new delivery formats. 

29 0 



296 /A Collection of Papers on Self-Study and Institutional Improvement, 2001 



Other Impact on Strategic Planning 

In addition to providing the basis for today's Estrella Mountain Strategic Plan, the 1996 self-study also had an impact 
on several other planning areas. Some of the changes that Estrella Mountain made based on self-study findings 
include: 

o Update of college vision 
o Redevelopment of mission and purposes 
o Creation of core values 

o Enhancing of institutional effectiveness and academic assessment processes 
o Streamlining and clarifying all college planning processes. 



Lessons Learned 



□ Need for Development of Continuous Planning and Review Cycle 

The self-study can provide an excellent catalyst to improve college strategic planning. However, systems need to 
be put in place to ensure that the college continues to conduct regular self-examination and reviews and scans 
of the external environment to ensure that strategic planning remains current. The danger is that the ten-year 
accreditation cycle may cause institutions to waittoo long to revise and update their strategic plan. The plan should 
be updated at least every three years and be reviewed for progress annually. 

Estrella Mountain is more aggressive in its monitoring and review process. The Strategic Planning Steering Team 
and Leadership Council review progress of the plan monthly. The plan is continuously updated when changes in 
the environment occur, and it is formerly updated at least once a year. Environmental scanning is also conducted 
on an annual basis. 

□ Need for Strong Linkages to Decision-Making and Resource Allocation Processes 

Without a link to decision-making and resource allocation process, strategic planning will not be effective. It is also 
important to communicate this linkage to college stakeholders because it increases the buy-in into the planning process. 

At Estrella Mountain the Strategic Directions and Institutional Priorities guide budget priorities for the College. To 
strengthen the linkage between planning and resource allocation, Estrella Mountain is currently updating all 
divisional and operational plans with improved linkages to the Strategic Plan. These updated plans will be a 
prerequisite for all future resource requests. 

□ Linkages to Assessment Efforts 

College assessment efforts should be linked to strategic planning and the resource allocation process. Colleges 
should examine how their institutional effectiveness, program review, and assessment of student learning result 
in requests for resources, improvement activities, and changes in practice. Estrella Mountain has made great 
strides in developing continuous improvement systems that result in concrete strategies and implementation; it 
is currently working on improving the linkage to the resource allocation process. 

□ Collaboration 

Planning is less successful if it is conducted in a vacuum. A highly participatory process involving many stake- 
holders results in increased buy-in and understanding. 

At Estrella Mountain Collaboration and Team Work is one of the college's Core Values. It has also been a part of 
every planning effort the College has undertaken. Most recently, the process of divisional planning has shown a 
need for interdivisional collaboration to ensure the effectiveness of the effort. 



□ Next Steps 




Estrella Mountain intends to build on the success of its first self-study and strengthen the linkage to strategic 
planning. The College's current self-study process will be completed by the end of summer 2001 . During the fall 
of 2001 , the College will use the findings from the self-study to update the Strategic Directions and Institutional 
Priorities for the college. 
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Estrella Mountain will maintain its commitment to monitor, review, and update the Estrella Mountain Strategic Plan 
on a continuous basis. The college will also work to develop systems to ensure that ongoing assessment systems, 
such as institutional effectiveness and student learning assessments, have stronger linkages to the planning and 
budgeting cycle. 



Clay Goodman is Associate Dean of Occupational Education at Estrella Mountain Community College in Avondale, 
Arizona. 

Rene Wi lie kens is Director of Institutional Planning and Research at Estrella Mountain Community College in Avondale, 
Arizona. 
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Back to the Future: Using the Self-Study 
Process to Facilitate Institutional 
Improvement Through Strategic Planning 



Charles Taylor 
Krystal Compas 
Wayne Chipman 

In the midst of all the millennial hoopla, 2000 was indeed a significant year in the 127 year history of Drury College. 
It saw a change in name and status to Drury University. It also occasioned a visit from the North Central Association. 
Our challenge, beyond the obvious preparations for meaningful self-study and a successful site visit, was to capitalize 
on the momentum established by a decade long period of unprecedented accomplishment in order to position our 
institution for the much ballyhooed new millennium. 

While we took nothing for granted, we were confident in the ultimate outcome of our NCA reaccreditation process. 
As a consequence, we were motivated to move beyond the minimally sufficient requirements of reaccreditation 
preparation to embrace an opportunity for campus-wide collaboration on crafting a strategic blueprint for the next 
decade. Drawing active participation from all campus constituencies, including faculty, staff, administration, students, 
and members of our Board of Trustees, our efforts resulted in a very positive site visit report and continued accreditation. 
Perhaps more importantly, we produced a detailed strategic plan with five action-oriented goals, a new commitment to 
working together to advance our institutional mission, and a deeper sense of campus community. In what follows, we 
will use our experiences as a case study to illustrate more general conclusions about the importance of meaningful 
strategic planning and the efficacy of integrating it with mandated self-study and reaccreditation responsibilities. 



Looking Back: Drury University 

Established in 1 873 as Drury College, Drury University is an institution of approximately 4450 students. The traditional 
day school program for traditional students enrolls 1431. The College of Graduate and Continuing Studies serves 
nontraditional students, many of whom attend part-time. Undergraduate enrollment is 2507, and 348 students are 
enrolled in five graduate programs. Drury is a member of the Associated New American Colleges, a national 
consortium of medium-sized universities with a commitment to the values of the traditional liberal arts college while 
offering a larger array of programs than many small colleges are able to provide. Drury maintains a student-faculty 
ratio of 14:1 and places an emphasis on excellence in teaching and individualized learning experiences. Drury offers 
accredited programs in business, education, and architecture, and is committed to the integration of liberal arts and 
professional preparation in all of its academic programs. 

The 1990s were years of significant advancement for the university. In addition to our name change, we invested 
significantly in new buildings and major renovations, including a state-of-the-art technology center, staffed 24/7. We 
added master’s programs in communication, criminal justice, and criminology. FTE enrollment grew by 33.2 percent, and 
the endowment grew from $26 million to approximately $100 million. On one level, then, it is difficult to argue with the 
relative success of Drury’s planning procedures, which have historically been coordinated through the president’s office 
and which have integrated all vice-presidents and major unit heads. The proof is in the pudding, as it were. 

Nonetheless, there was some concern that faculty and staff input, while welcomed, may not be integrated 
meaningfully into long-term institutional [read: budgetary] decision making. This had led, in some cases, to a measure 
of skepticism among faculty and staff regarding the utility of participating in planning processes. Inclusion is a key 
principle of Drury’s strategic planning, but there was a lingering sense that while planning is ongoing, it was difficult 
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to see closure for those efforts. In effect, planning was seen as just another “black hole” into which faculty and staff 
time and energies vanished. If, in fact, perception shapes [or overdetermines) our institutional reality, our challenge 
was to eliminate the disconnect between the two. 



Back to the Future: The Self-Study Process and Strategic Planning 

In the remarkably dynamic, even unstable, landscape of higher education, resting on one’s institutional laurels is a recipe 
for decline. To meet the changing demands on our institutions, it is essential to creatively evaluate and respond to that 
landscape. The key, then, is to find ways of “arraying options through a process of opening up institutional thinking to 
a range of alternatives and decisions that identify the best fit between the institution, its resources, and the environment" 
[Rowley et al. 1 997, p. 1 5 ). The NCA Accreditation Criterion Four Committee Report underscored the significance of this 

way of institutional thinking: “Continuous strategic planning thatisfutu re oriented, fosters dynamic programming and 

is based on sound financial and operational strategies is the key to maximizing the college’s strengths and is a basis 
for ensuring that the college can continue to meet the needs of its constituency” [Drury University 2000, p. 5). 

Dubbed “NCA2000” as a reflection of the integration of self-study preparation and long-term strategic planning, our 
strategic planning team began its efforts in summer 1998. Responding to a call from the VPAA to all campus 
constituencies, a team of 35 members met and developed a guiding vision for the future, “to build the best Collegiate 
University (New American College) in the region,” and strategic directions that could make that vision a reality. 
Subcommittees (constituted intentionally with representation from faculty, staff, and student populations) were 
assigned the responsibility of crafting individual sections of a meaningful SWOT analysis that focused on the 
strengths, weaknesses, opportunities, and threats facing Drury University. 

While the SWOT approach has much to recommend it, care must be taken to ensure that a candid assessment is 
produced. Identifying strengths and opportunities might come naturally to a group of faculty, staff, and students who 
are committed to the institution, but noting weaknesses (easier for students than faculty, we found) and threats 
represent larger challenges. In part, ego involvement is an obstacle. Indeed, the subcommittee charged with 
identifying weaknesses initially rewrote their charge to focus on “unrealized opportunities” until they recognized that 
self-reflection requires scrupulous, if occasionally painful, honesty. Additionally, the relatively short-term focus of 
much staff, faculty, and student thinking (e.g., the next semester or project) does not lend itself easily to the longer 
term strategic perspective required. Judicious recruitment of persons with appropriate skills— when self-selection 
fails-is an important consideration. 

Using the results of the SWOT analysis as a foundation, the NCA2000 team coordinated a series of focus group 
sessions with various campus units and constituencies. Utilizing trained facilitators, participants were asked to 
brainstorm potential goals and then to prioritize them in terms of overall importance to enactment of the vision 
statement. The sessions, which were held at an annual faculty retreat, staff meetings, and special student meetings 
(coordinated with the cooperation of Drury’s Student Senate), provided multiple forums for input from across campus. 
This functioned not only to generate invaluable data, but also to reaffirm the sense of community we value across 
campus. The necessity for prioritizing the goals encouraged a consensual transcendence of narrow partisan interests 
forthe larger good of the institution. In addition, new information technologies, including an interactive NCA2000 web 
site, were available for campus-wide input. 

Five major goals emerged from this extensive series of focus group meetings and subsequent reflection. The NCA2000 
group refined the language in which the goals were expressed into the following: 

1. To strengthen academic programs and add selected programs. This would include planning in the 
areas of faculty (teaching excellence, faculty development, scholarship, salaries, increased technological 
abilities, etc.), student learning (increased f 0C us on technological literacy and student scholarship), and 
improved retention and selective addition of new programs. 

2. To strengthen the sense of community (a learning community). This would include planning intention- 
ally to maintain and strengthen the sense of community, the traditions, the ways in which people connect, the 
student-faculty ratio, student-staff ratio, the contacts between students and faculty (with improved commu- 
nity and services for commuter students and nontraditional students). 

3. To increase the diversity of educational experiences available to students. ConsistentwithGP21 (Drury’s 
general education curriculum) and the mission, planning will focus on creating a place where diversity of persons 
and ideas is welcome; increasing international study opportunities and participation; increasing the number of 
international students and faculty; and increasing the number of American minority students, faculty, and staff. 
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4. To enhance the integration of liberal arts and professional programs. As a collegiate university, one 
of the strengths is the preparation of students for successful professional careers [the work of all majors], and 
that preparation is best when it is carefully infused with the liberal arts. The planning here will focus on 
developing distinctive programs with clear outcomes that achieve this goal-programs that are continuously 
improved through assessment. 

5. To provide the infrastructure to support growth and other goals. To pursue these goals will likely result 
in improved retention and growth in both traditional and nontraditional students. Conversely, managed 
growth can contribute to the achievement of these goals. Planning will focus on endowment growth, facilities 
development, technology, administrative support, staff development, etc. 

Action teams were organized around each of the articulated goals and were charged with developing specific plans 
for achieving the goals and developing budget proposals for submission to the administration and university budget 
committee. Once again drawing representation from all campus constituencies, teams met throughout the academic 
year, producing a formal, bound document ratified by faculty, staff, and Board of Trustees action [Defining the 
University, 2000]. 

Significant action has already been motivated by the strategic planning process and was evident during the 
September 2000 NCA site team visit. For example, the commitment to strengthening academic programs was realized 
in small part through the creation of a Center for Faculty Excellence. The Center, which opened in fall 2000 with a new 
full-time director, is charged with providing faculty development for enhanced teaching, scholarship, and service. 
Similarly, to address the commitment to diversity and international educational experiences, Drury has recently 
affiliated with the AN ACSA [Associated New American CollegesStudy Abroad] program. This affiliation increases Drury 
student access to well-established and intellectually rewarding international study experiences. In addition, discussions 
are underway about the feasibility of establishing a Drury University International Study Center in Greece. 

The next important step is for the plan to become institutionalized, ensuring the perception of closure that has eluded 
previous strategic planning initiatives. Toward that end, the VPAA has begun the creation of a strategic planning 
council, to be composed of the president, four vice presidents, three elected faculty, two students, and three staff 
members. The council will be charged with: 

1. Overseeing the development of specific budget proposals framed by the strategic plan goals. 

2. Reconciling conflicts between the plan and invariably limited resources. This will include development of a 
long-range budget that supports the strategic plan. 

3. Developing a pattern of regular review of the implementation of the strategic plan. This will include monitoring 
reports from those accountable for various aspects of the plan and providing feedback. 

4. Participating actively in the budgeting process, prioritizing budget requests in terms of their relevance to the strategic 
plan and importance in the plan. While a number of individual budget items for AY 2000-2001 were products 
of the NCA2000 plan, this will be a comprehensive process for deliberations affecting the 2001-2002 budget. 

5. Regularly checking the process of the institution against relevant benchmarks [appended to bound copies 
of the plan]. 

6. Regularly reporting in formal and informal contexts to the institution’s stakeholders on progress toward 
achieving the plan’s goals. 

As work proceeded on the various dimensions of strategic planning, self-study preparations were also underway. 
Self-study chapter team chairs met regularly with the NCA2000 team, and the self-study steering team included 
members from the larger group. This helped facilitate efficient and meaningful information exchange. The self- 
reflective character of the self-study was informed by the significant data generated through the strategic planning 
process, and that process received important direction and a productive sense of urgency from the exigencies of the 
self-study process and impending site visit. 
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Making the Most of 
Consultant Assistance in Colleges 



Marie A. Giacomelli 
M. Jane Hunter 
Carroll Bennett 

External assistance can be extremely helpful in moving an institution forward. Using consultants can often be the 
catalyst for colleges achieving the excellence and quality they seek. Consultant assistance can be a valuable 
educational resource; and, as such, its use deserves careful consideration in the same way any of the institution’s 
resources (physical, financial, human] and processes are planned, implemented, and evaluated. 

Any college can profit from an outside, objective view furnished by a consultant. If a college is small, it often needs 
to use outside help to get certain things done. If a college plans major changes or faces major challenges, it usually 
needs outside help. If a college has an outstanding reputation, a consultant can help it reach even higher goals. 

Consultant assistance can be engaged through a variety of sources: recognized consulting firms, professional 
organizations, peer institutions, or freelance/independent consultants. And, even more varied are the general 
purposes for which the institution might engage a consultant's services; for instance: 

o To help define issues, suggest approaches, provide third-party analysis of work in progress, furnish a point 
of view, or establish priorities for action. 

o To supply a broad perspective on external trends pertinent to the growth and development of the institution. 

o To identify other institutions that can serve as an informal benchmark or that have successfully solved similar 
issues. 

o To audit policies and practices for congruence with or to distill core values. 

o To integrate existing and innovative practices into a new, more effective framework. 

o To facilitate group interaction or provide training regarding a defined topic or specific operational aspect 

Regardless of the general purpose, the consultant— as an outside observer— brings a fresh perspective and knowledge 
to escalate learning and action within the organization. Furthermore, the combination of external consultant 
assistance and internal faculty-staff expertise can be especially productive because of the range of insights brought 
to bear on the matter at hand. 

Deciding to enlist consultant assistance merits planning at the outset and oversight during the process. The working 
relationship forged between the institution and the consultant can be as formal or informal as both parties find 
comfortable. Such assumptions will influence how the following suggestions apply to the arrangement. 



Establishing the Consultant-Client Relationship 

Any institution considering the use of a consultant is faced with many questions that are critical at the outset Foremost are: 

1 . Why is the use of a consultant appropriate? 

2. What are the expected outcomes Of the consulting process? 

3. How will the assistance be provided? 
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The answers to these questions should be formulated before initiating the search for a consultant to provide the 
service. In grappling with these questions, the institution may wish to consider the following approaches. 

The use of a consultant is appropriate provided the institution retains overall responsibility for the issue or process 
for which assistance is being secured. While consultants can bean invaluable asset insofar as guiding and facilitating, 
they should not be responsible for directing a project. One key to accomplishing this is the institution’s commitment 
to viewing the consultant’s help as a formative tool rather than a summative one. Such a strategy is required especially 
if the consultant is hired to accomplish a task college staff do not have time or requisite skills to undertake. 

The institution should clearly define its needs. The expected outcomes of the consulting process should bp articulated 
in a written project statement that includes purposes and objectives. The importance of this step cannot be overstated. 
The expected outcomes will give direction to selecting a consultant and delineating the specific responsibilities of 
each party involved in the project. The format, schedule, and dynamics for the process as well as negotiation of the 
consulting agreement will largely be dependent on the needs definition. The productivity of the consultant and the 
process will be impeded in the absence of such a statement and the conceptualization it requires. 

Selection of a consultant should address many of the same basic considerations made in the selection of new faculty 
or staff members for the institution. The project description, purposes, and objectives inform the consultant selection 
process in much the same way a job description serves the employee selection process. Compatibility between the 
institution’s needs and the prospective consultant’s skills is essential. The institution may wish to seek referrals from 
colleagues who have used a consultant for the same or a similar purpose. Interviewing three or four candidates is 
helpful in order to compare their experience, knowledge, and references. The candidates’ ability to interact well and 
communicate results in writing is especially important for projects where the consultant will be conducting individual 
and/or small group meetings. 

Other practical aspects of the working relationship between the college and the consultantdeserve detailed attention 
during the selection process if misunderstandings are to be avoided. Obviously, fees, expenses, terms of payment, 
and what services will be rendered should be agreed on. How the consultant will engage with the institution, the 
timeline, and the form for the deliverables should be decided. Like the project definition statement, such details should 
be summarized in a formal or informal memorandum of understanding before the project begins. 



Implementing the Consulting Process 

Both the institution and the consultant carry responsibility for maintaining a collegial working climate throughout the 
project. Open-mindedness on the part of all participants is a necessity; therefore, time devoted to discussions among 
the institution’s participants and with the consultant to “set the stage" is an investment in the eventual success and 
outcomes of the process. Mutual agreement about the respective responsibilities forms the foundation for the 
consultancy. 

The institution should treat consultants as trusted professionals and be mindful that they lack the inside information 
crucial to understanding the inner workings of the college. The institution should provide an extensive orientation on 
its needs, furnish support materials at the start, and make additional needed resources available as the project 
evolves. The consultant must treat all information as confidential and not divulge any confidences that are shared by 
the institution. 

The consultant should respectthe college’s statement of the problem or its needs, recognizing that institutional values 
influence the institutional climate and vice versa. The consultant must understand and be able to relate to the 
institutional culture, history, constituencies, current activities, and future plans. While the consultant can glean some 
of this information from written documents, the institution should encourage broad faculty/staff involvement in a 
dialogue with the consultant to capture their collective views and experience. 

The consultant’s most important contribution is bringing an objective view. Most consultants have visited many 
colleges and have seen processes that work well and others that are less effective. They can carry the word on good 
practices and caution against procedures that are less likely to work or perhaps even to fail. Their advice should not 
be perceived by the college’s participants as an intent to reject old paradigms or to create defensiveness. 
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As an outsider, it is possible for a consultant to see contradictions, disconnects, or omissions in institutional 
documents and procedures that are not apparent to college personnel. Likewise, the consultant can observe 
institutional actions or decisions undertaken as a matter of course that are ineffective or inappropriate. Another 
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hindrance to institutional progress with which the consultant can be helpful is the “we’ve always done it that way” 
syndrome; internal constituents are often too close to the situation to be able to identify the problems inherent in the 
current mode of operation. 

Consultants can be more candid with institutional personnel than can people within the college. The consultant can 
express concerns that peers or leaders on campus are hesitant to verbalize to each other for fear of “rocking the boat.” 
The consultant may reserve findings of a particularly sensitive nature for a special report to the president. 

Designing a schedule for the consultant’s time on campus is a natural extension of the project statement provided 
the statement is explicit about the institution’s needs and desired outcome. A project coordinator designated by the 
institution should propose the individuals or groups with whom the consultant interacts in order to exchange 
information related to the consulting topic. For the most efficient use of on-campus time, the coordinator and 
consultant can collaborate about the kind and distribution of introductory and support materials that are needed in 
advance by the institutional participants and the consultant, respectively. Participants’ comfort with the process and 
confidence about it are usually in direct proportion to their understanding of the project purposes and objectives. 



Evaluating the Project 

Depending on the length and scope of the consulting agreement, the college and consultant should periodically 
assess the progress being made toward project objectives. Feedback from the participating staff and the consultant 
about how well the arrangement is working can be used to make schedule adjustments or other revisions. 

The institution should not view the consultant’s report as the end of the process, but rather as the beginning of the 
second half of the process. The follow-up activities in which the college chooses to engage, based on the consultant’s 
report, will determine the overall value to the institution of having hired a consultant. The institution’s commitment 
to follow-up should be made in the initial planning for the project, even though identifying the follow-up tactics most 
likely can best be determined after studying the consultant’s report. 

On completion of the project, many consultants request that the client institution provide an evaluation of the services 
received. The assessment can be useful to both parties. For its completion, the institution will need to reflect on the 
project’s planning and logistics. Such a retrospective can assist the remainder of the process and will aid in designing 
any future consulting arrangements. Good consultants will regard the evaluative comments in terms of improving 
their services; of course, they appreciate testimonials, too! 

Over the past decade, Robert Morris College has planned and implemented a series of successful institutional 
changes, including the addition of a new degree level and new instructional sites. With the overall goal of achieving 
balance between quality and growth, the institution used consultant assistance as one of its quality-review strategies 
during the course of internalizing the changes. Consultants were engaged to offer constructive insights about diverse 
topics: curriculum, instructional modes, student support services, the assessment program, library resources and 
services, long-range planning, and faculty life. 

Three stated purposes guided the project-dubbed the Institutional Quality Task Force-and the collective input 
garnered through the participation of faculty, staff, administrators, and consultants: 

o To obtain an external perspective of the college’s performance in relation to institutional mission, institutional 
viability, and commonly accepted standards of good practice in higher education. 

o To identify opportunities for enhancement and suggestions for pursuing them based on observed institutional 
strengths. 

o To furnish information useful to the steering and working committees in conducting the institutional self-study 

for continued accreditation. 

The factors and general suggestions described in the above article were applied to arranging the consultant services 
and guiding the dynamics of the process. 
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The Experimental Approach: 
Planning as Continuous 
Quality Improvement 



Jon Uuistgaard 
Michael Field 
M. James Bensen 
Clyde Jacobs 
Ivan Weir 



Introduction 

Bemidji State University has long been engaged in, and has recognized the value of, planning in accomplishing its 
purposes. Like many institutions, it has used a variety of approaches to accomplish planning. Many of the approaches 
implemented over the years have fallen short of expectations in achieving the desired alignment between the 
institution purposes, resource allocations, and the changing environment. The institution has a history of uneven 
success in its attempts to initiate meaningful long-range planning. Rather, the tendency has been to focus on short- 
term operational responses to complex problems. The ten-year comprehensive re-accreditation visit provided an 
opportunity for the university, in cooperation with the North Central Association, to engage in an Experimental 
Planning Initiative as part of its self-study and site visitation process. Bemidji State University’s objective was to 
develop and implement a strategic approach to planning. This presentation will focus on (1) the identification of shared 
commitments and organizational principles necessary for meaningful planning in a collective bargaining environment; 
(2) a description of the university’s empowerment orientation in an environment characterized by change; (3) the 
identification of working assumptions through an analysis of institutional strengths, weakness, opportunities, and 
threats (SWOT); (4) a discussion of the principal structural planning recommendations; and (5) a description of several 
representative examples of the planning outcomes achieved as a result of the Experimental Planning Initiative. 

The thoughts and developments that have resulted from the implementation of the Experimental Planning Initiative 
continue to evolve through a deliberative process that has encompassed nearly three years. Additional planning 
expertise beyond the university in the formulation stage was provided by consultants from the University of 
Minnesota-Hubert Humphrey Institute for Public Policy, the North Central Association, Minnesota State Colleges and 
Universities System (MnSCU), and several other universities. 



The Value of Dialogue: Shared Organizational Commitments and Principles 



In collaboration with the bargaining units, a series of community-wide dialogues were convened with nearly every 
university employee. Members of the university’s student senate, foundation, and alumni association; business; and 
community also participated in the dialogues. The purpose of the informal gatherings was to discuss university 
planning for the twenty-first century. The key questions that were an outcome of these discussions fit very closely 
with the strategic planning questions outlined by Rowley, Lujan, and Dolence in Strategic Change in Colleges and 
Universities: Planning to Survive and Prosper (1997). They are: (1) Who will our students be? (2) What should students 
learn? (3] What opportunities maximize student learning during the college experience? (4] What resources will we 
need? (5] How will we provide the needed resources? (6) What do our constituents expect of Bemidji State University? 
(7) How will we know when our purposes are being accomplished? (8) How will we tell our story? (9] How will we 
collaborate with our partners? 
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Another outcome of the planning meetings was the identification of both shared commitments and organizational 
principles. Many of these foundational statements resemble those described in the writings of Peter Senge, most 
notably in The Fifth Discipline 0994). Included in the shared commitments were comments such as 0) a faculty, staff, 
and administration devoted to student achievement and success; (2) a learning community empowered by the 
applications of technology to facilitate the learning process; (3) a university enriched by native heritages and world 
cultures; and [4) an involvement in innovative partnerships with education, business, and industry for advancing the 
educational, cultural, social, and economic interests of Minnesota, most particularly the northern tier of the state. All 
of the commitments identified by the participants are affirmed in the university’s vision statement and mission statement 

In addition to the identification of shared commitments, six organizational principles were identified. Among these 
were 0) a recognition that broad-based participation in developing planning approaches and strategies through 
shared personal experiences, knowledge, and imagination is important; (2) an interest in having increasing 
opportunities to interact in an empowering environment that encourages and enables experimentation with new and 
different ideas; and (3) a genuine interest in valuing colleagues, regardless of assignment, for their contributions to 
learners and the university community. 

Both the shared commitments and the shared organizational principles have been used as the foundation for the 
development of a climate conducive to implementation and maintenance of the Experimental Planning Initiative. 



Empowerment 

Peter Senge and his learning organization colleagues suggest that “meaningful change in institutional culture means 
redesigning not just the formal structures of the organization, but the hard-to-see patterns of interaction between 
people and processes” [Senge, 1994). A sense of genuine empowerment is more likely to occur when lines of 
communication are open and institutional hierarchy is not reinforced by procedural requirements that may be 
unnecessary or overly complex. This approach to empowerment turns traditional models of hierarchical organiza- 
tional structures upside down. Such a shift in attitude and institutional responsiveness places students at the top of 
the hierarchy. 

The empowerment theme is central to the Experimental Planning Initiative. It was particularly important in 
reconstituting the planning structures. A sense of “real empowerment” was fostered when the planning committees 
received more autonomy and authority in the planning process. This does not imply that committees have ultimate 
authority, but that, except in unusual circumstances, they can reasonably expect their recommendations to be 
supported; if their recommendations are not accepted, they will receive a prompt and thorough response and 
explanation. We learned early in the community-wide dialogues that a universal desire exists to become increasingly 
knowledgeable about the “thinking of the university.” This position has been validated throughout the planning 
implementation process. 



Planning Assumptions 

The university is moving rapidly toward the adoption of a strategic thinking approach to planning. An approach based 
on clearly communicated institutional purposes is grounded in an understanding of the interrelationship between the 
institution, its constituencies, and the environment. The Experimental Planning Initiative is grounded in a number of 
working assumptions derived from an analysis of strengths, weakness, opportunities, and threats in both the external 
and internal environments. The characteristics identified by the community-wide dialogues are very closely aligned 
with Pew Funded Projects focusing on institutional change and transformation. These challenges include [1) the 
pressure to contain cost and keep higher education affordable; (2) public demands for educational and financial 
accountability; [3) increased demands for educational quality and excellence; [4) growth of alternative models of 
postsecondary educational delivery, including distance education, corporate universities, and transnational delivery; 
[5) the explosion of knowledge produced both inside and outside the academy; [6) the need to serve an increasingly 
diverse society; (7) the pervasive impacts of technology on all areas of higher education. Additional external 
challenges encompass membership in a new higher education system [i.e., Minnesota State Colleges and Universities 
System [MnSCU), legislative priorities, demographics, and private sector and public sector needs). 
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The internal SWOT identified seven planning assumptions to be considered in the development of the Experimental 
Planning Initiative. Included among these assumptions were (1) placing learning as the university’s first priority; [2) 
institutional planning fully consistent with all collective bargaining agreements; [3) planning that embraces ongoing 
self-assessment procedures and feedback mechanism; (4) planning guided by shared commitments and shared 
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principles; [5) continuing fiscal constraints affecting the university’s ability to offer educational services; [6) societal 
expectations requiring flexibility in meeting the needs of the learner, business, and industry; (7) in addition to the 
university’s primary purpose of providing learner access to high quality educational opportunities on campus, the 
recognition that partnerships will increasingly create new and expanded learning opportunities. 

The seven planning assumptions derived from the SWOT analysis continue to guide the evolution of the Experimental 
Planning Initiative. SWOT analyses are performed at several points each year for purposes of accommodating change 
to better alignment of the institution with the internal and external environments. However, unlike previous planning 
approaches, the focus is now on understanding the implications of a rapidly changing context from the perspective 
of accomplishing the university’s longer-range purposes. 

Three Recommendations for Planning 

Three principle planning recommendations were identified as critical to the university if it was to better realize its 
potential and purposes. Although the principles are simply stated, the realization and maintenance of an empowering 
environment is an ongoing challenge. I he three principal planning needs are as follows: 

1. The university needs to clarify and reaffirm the primacy of its academic mission, setting clear institutional 
priorities and demonstrating that those priorities govern its decisions as commitments above all to students 
and learning. 

2. The university needs to simplify procedures and institutional processes at every level, while preserving and 
enhancing meaningful communications, consultation, and capacity for timely decision making. 

3. The university needs a leaner committee structure that incorporates a major planning committee responsible 
for coordinating planning efforts and recommending appropriate actions. 

To provide a structure for realizing these recommendations, three major university-wide councils, including a 
University Council intended to serve as the primary body for institutional planning, have been created. The University 
Council consists of the co-chairs of the six university-wide planning committees. The Council is co-chaired by the 
Vice President for Academic and Student Affairs and the Vice President for Administrative Affairs. The co-chairs share 
responsibility for communicating with the bargaining units. While the University Council has many functions and 
responsibilities, its primary function is institutional planning. The university-wide planning committees O.e., academic 
affairs; budget and resource allocation; computing, technology, and learning resources; outreach and partnerships; 
recruitment and retention; and student and university services) are responsible for developing planning initiatives 
following consultation with the campus community for consideration by the University Council to accomplish the 
university’s purposes. The six university-wide planning committees have replaced nearly twenty university commit- 
tees. In addition to the University Council, an Innovations Council and Executive Council have also been created. The 
President chairs the Innovations Council. Its membership is drawn principally from representatives of business, all 
levels of education, civic and political leaders and special constituencies. The purpose of the Innovations Council is 
to advise the President on leading-edge developments to keep the university positioned as an educational innovator 
and focus of creativity and excitement. As such, it is intended to be a continuing source of new ideas with an emphasis 
on refining the university’s vision. 

The third council, the Executive Council, is composed of the President, the Vice Presidents, and the Deans. The 
Executive Council advises the President on matters pertaining to university policy, including consideration of planning 
initiatives proposed by the university-wide planning committees and the University Council. The Executive Council 
composition represents a strengthening of the university’s commitment to the principle of academics through the 
inclusion of the deans. 

The three Councils represent a structure that facilitates relationships and communications between planning, 
innovations, and policy implementation. The organizational structure also provides mechanisms and communication 
channels across the Councils and throughout the university community while respecting collective bargaining 
agreements. 

Planning Outcomes 

The Experimental Planning Initiative is in its third year. The new planning committees are fully implemented. Planning 
initiatives are moving forward from the planning committees to the University Council. Each planning initiative is 
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linked to the university’s mission. The University Council continues to provide planning recommendations to the 
Executive Council. The Executive Council has acted on all recommendations and communicated to planning 
committees, the University Council, and campus community the results of its decision-making process. Planning 
initiatives that have been enacted include the preparation of a university strategic plan, a comprehensive upgrade 
of the campus computer network, and the expansion of five-year program reviews that include a formal assessment 
component for all student and administrative service areas. 

As the institution moves forward with its strategic planning, it is guided by the thoughts of Rowley, Lujan, and Dolence 
who stated in Strategic Change in Colleges and Universities that: 

For a strategic plan to be effective and adequately affect policy, it must state the obvious directly, address the 
resulting issues simply and understandably, and identify the niche of opportunities within the grasp of the 
institution, given its role and capacity.To reinstate the obvious planning principle, the strategic plan must gradually 
become an instrument that, in spite of its inelegance, unites rather than divides the campus. 0997, p. 302] 



Summary 

The Experimental Planning Initiative is increasingly a success story. Beginning with the initial series of community 
dialogues, it was evident the institution was prepared to embrace change under the appropriate circumstances. The 
process to proposing the Experimental Planning Initiative has been somewhat lengthy and arduous. The comprehen- 
sive self-study site visit was very important in creating an institutional will to undertake yet another attempt to create 
a meaningful approach to planning that would be embraced by the campus community. Leadership provided by 
several key senior faculty members and encouragement by external consultants and the chair of the North Central 
Association visitation team were critical to the validation of discussion points and proposals. Questions raised during 
the self-study visit were helpful in clarifying continuing challenges and in encouraging the institution to move forward. 
The test of time will ultimately validate both what has been proposed and what has been accomplished. The plan and 
its intended outcomes are not without its skeptics. Yet, there is a climate of increasing optimism that the strategic 
planning process and the strategic plan itself provide the necessary opportunities to continue the university’s rich 
tradition of providing education that enriches and transforms the lives of its learners. 
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The Learning Organization as a 
Model for Successful Self-Studies 



David Porter 
Thomas A. Angelo 

Our title begs at least three key questions. What is a successful self-study? What is a learning organization? How can 
they be linked productively? Though we’ll focus most of our attention on the last two questions, it’s important to define 
success first. 



Defining a Successful Self-Study 

We consider gaining reaccreditation a necessary but not sufficient goal for the self-study process. An institution that 
treats self-study and related assessment activities as simply pro forma compliance exercises is missing opportunities 
for sustained, meaningful organizational learning and potential renewal. From our point of view, a successful self- 
study both creates and capitalizes on such opportunities. And we’re not alone in this. The regional and professional 
accrediting agencies, including NCA, a|l urge institutions to make their self-studies internally meaningful. 

For us, then, successful self-studies are authentic and potentially transformative. They differ from traditional, pro 
forma self-studies that seek to create only the illusion of perfection. Authentic self-studies are aimed at comprehen- 
sive and continuing institutional improvement; they may require changes of consciousness, even shifts in paradigm. 
In contrast, traditional approaches are often driven by the desire to meet particular, predetermined, external criteria, 
to “answer the mail” or “check off the boxes,” and, in so doing, earn another 10 years of accreditation. Such inauthentic 
approaches are likely to frustrate and alienate important constituencies within the institution and obscure rather than 
reveal opportunities to improve faculty and student learning. An authentic self-study asks questions that are 
consequential, that will make a difference to the process of teaching and learning. It requires a willingness to recognize 
extant problems and discontinuities and a commitmentto address them. While not every institution wjll choose to engage 
in such authentic, transformative self-studies, those that do can reap substantial and enduring benefits. 

Enhancing educational effectiveness and quality through the self-study requires developing shared goals and increased 
trust. In other words, it requires a change in the culture. Learning organizations offer higher education a framework for 
self-studies that can help colleges and universities achieve such change. To begin with, conducting an authentic self- 
study requires the institution to select and focus on goals worthy of the time and effort required to prepare any study. 
The self-study goals must be widely shared and valued! Making the president, provost, or board look good is a goal 
unlikely to motivate many faculty and staff, for example. The next step is to actively involve the entire campus in 
developing a shared vision-among individuals representing all constituencies^ what the institution is striving to 
become. Not everyone has to participate for the self-study to be an authentic success, but everyone must be invited. 

Another key component of a successful self-study is the development of a truly effective assessment process. Unlike 
its cosmetic imitations, authentic assessment, like science itself, is anchored in the principle of falsifiability. This 
suggests that assessment will be so bold as to seek evidence contrary to the institution’s own assumptions and 
aspirations. Every academic policy, every administrative practice, every curriculum and pedagogical approach can be 
considered as hypotheses, propositions begging to be tested against the patterns of performance they produce. 



Developing a Learning Organization 

Learning organizations are groups in which individuals work together to create systems that adapt and thrive and in 
which individuals have increasing opportunities to take greater pride in what they produce. The late W. Edwards 
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Deming, founder of the integrated set of principles known as “total quality,” provided a cogent approach to effecting 
such transformational changes (Deming, 1986). Deming was an educator. He believed that “profound knowledge,” 
which included an awareness of statistics, psychology, systems, and the nature of knowledge itself, was essential to 
authentic quality improvement. Until his dying day, Deming waged war against quality pretenders, those who 
emphasized techniques and appearance over substantial improvements to organizational systems. In the manufac- 
turing and service industries, Deming believed that achieving true quality necessarily and directly involved taking 
seriously the perceptions and motivations of workers themselves. Ironically, many educational institutions appear to 
have been tempted to neglect the theoretical aspects of Deming’s approach and to simply adopt some of the tools 
and techniques developed by his students and disciples. For these academic quality pretenders, self-study and 
assessment processes can become superficial activities designed to secure reaccreditation by creating an illusion 
of academic excellence. 

PeterSenge’s (1 990) five disciplines of a learning organization (personal mastery, mental models, sharing vision, team 
learning and systems thinking) are a refinement of Deming’s original concept of profound knowledge. Defining a 
discipline as a practice that builds capacity over time, Senge suggests that these five disciplines, taken together, can 
be necessary and sufficient to developing a learning organization. Each of these disciplines can help determine 
appropriate organizational policies and shape the processes of authentic institutional self-studies. 

The first discipline is persona! mastery. This discipline challenges each person in the organization to continually clarify 
and deepen her or his personal awareness, understanding, and ability. The process begins with identifying personal 
values and priorities, then making a sincere commitment to actively pursue these espoused goals. Personal mastery 
represents the grassroots of any learning organization; it challenges everyone to start “walking their talk.” This 
discipline also reminds everyone that each individual in the organization has something to learn. Traditional 
distinctions between the learned administrators and faculty and the student learners not only create artificial 
boundaries and unnecessary alienation, they diminish the chances that anything truly useful will be learned from the 
self-study process. 

The model below depicts the development of personal mastery. Referred to as the AOR model for its three component 
processes, it suggests that personal mastery involves and requires three successive activities: action, observation, and 
reflection (Hughes et al., 1999). Many individuals (as well as organizations) stagnate because they overemphasize 
a single component and neglect one or both of the others. 




The discipline of personal mastery has many implications for conducting an authentic self-study. It seems odd that 
in an industry supposedly dedicated to human development, there is so little actual inquiry into the learning process. 
Might the AOR model apply to classroom teaching and learning? Regrettably, teaching is often done in isolation, 
seldom observed, critiqued, or appreciated by either administrators or colleagues. Academic freedom is often the 
rationalization for this extraordinary level of privacy, but the underlying issues are usually fear and lack of trust. 
Perhaps it is the nearly complete absence of observation that renders the woeful inadequacy of reflection moot. 
Authentic assessment of higher education must start where the action is: in the classroom where we assume teaching 
and learning occur. Basic questions such as Who is doing what? How are they doing it? How well is it working? How 
do we know? beg to be addressed. Much of the recent success of academic audits rests in their capacity to initiate 
such inquiry across the institution. The same questions should also be applied to the training and education of faculty, 
staff, and administrators. 

One final but very significant implication of this discipline for self-study committees is to construct them so they will 
promote personal mastery among their members. Committees need opportunity, access, authority, and resources to 
address the institution’s most important questions. To expect these committees to meet significant challenges but 
then fail to provide necessary support clearly conveys the administration’s intent that assessment be more apparent 
than actual. 
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Mental models are the second discipline of learning organizations. These ubiquitous cognitive structures are 
constructed of our underlying assumptions and reflected by patterns in our behavior. Although individuals are often 
unaware of their own implicit “schemas,” these mental models guide perception and thinking as well as behavior. This 
discipline invites individuals to discover and examine their own assumptions. It involves real conversations that blend 
advocacy and inquiry, with each individual asking questions in order to learn and better understand, and stating views 
so they become increasingly comprehensible to colleagues. 



Douglas McGregor’s 0966) classic distinction between Theory X and Theory Y reflects alternative mental models. 
Theory X adherents assume that learning is work, that it is distasteful, and that students and faculty must be coerced 
or cajoled into performing necessary educational tasks. This theory often leads to a lack of institutional innovation 
and the inevitable attribution that the institution simply does not have the right kind of students, faculty, or staff. In 
contrast, Theory Y begins with the assumption that everyone wants to learn because it is intrinsically motivating. This 
theory often leads to institutional innovation and high levels of enthusiastic participation and adaptation. Unfortu- 
nately, many administrators and faculty members assert their adherence to a progressive Theory Y, but their policies 
and programs make their commitment to Theory X clear. Such discontinuities obviate the development of institutional 
as well as individual integrity and inhibit the growth of trust (Argyris and Schon, 1978). 

Mental models have important implications for the self-study process. Often instruments and assessment activities 
are selected without reference to any underlying theoretical model of education. Such atheoretical groping and 
hoping is likely to be both expensive and ultimately counterproductive. Deming referred to it as tampering (Aguayo, 
1 990). Every assessment should be supported by an explanation of what might be learned and how the results will 
be used. Seldom will a single instrument or assessment activity yield evidence that is so persuasive that it compels 
immediate action. However, without suitable theoretical models, results from different studies using different 
methods and instruments and diverse samples of subjects can never be synthesized. Authentic assessment is done 
for the purpose of testing hypotheses; pro forma assessment is done for the purpose of impressing others (either with 
results or sometimes with the mere investment in activity). The most obvious way to distinguish the two is to ask those 
conducting a study to share their mental model concerning what the instrument is measuring, what they hope to learn 
from the results, and how the results will be used to improve the educational process. 

Sharing vision is the third discipline. Given a choice, most people aspire to important, even ambitious, goals (at least 
according to Theory Y). What galvanizes an organization in the pursuit of excellence is the process of having 
substantive conversations about the future each individual seeks to create. The practice of sharing vision involves 
revealing one’s deepest desires and most significant aspirations; it fosters genuine commitment and mutual trust. 
Sharing a vision is hard work. To be effective, it must be practiced continuously rather than once every ten years. In 
its simplest form, it might involve asking each individual and each constituency what they have done that contributed 
the most to institutional goals in the recent past, what they hope to do in the near future, and what support they need 
from others to be successful. There is great value in having each individual talk about what he or she is looking for 
and what the environment would look like as the institution made progress toward achieving its purpose. Sharing the 
vision is the primary means for sharing the patience as well as the passion that characterize highly effective learning 
organizations. Sharing vision is also one of three components of Peter Senge’s (1999) "Innovation Eco-system.” 
Learning (and relearning) the institutional purpose provides the foundation for vision sharing. Assessing results is 
a direct consequence of vision sharing. The ongoing pursuit of these three activities is the cycle of innovation that 
provides institutions with increasing adaptive capacity. 



Basically, sharing vision is what an authentic self-study does; it’s implications for conducting self-studies are 
profound. Authentic self-studies are likely to follow Senge’s cycle of innovation. Far too little attention is paid in most 
self-studies to discerning the institution’s purpose. Some institutions have rambling mission statements that basically 
commit to being all things to all constituencies all the time. Others have statements so succinct but ambiguous that 
no one even attempts to explain or apply them. One of the first goals of self-assessment should be to help the 
institution reach a common understanding about its unique mission and priorities. Self-study committees provide an 
ideal venue for the conversations characteristic of authentic vision sharing. Both of these activities are necessary 
prerequisites to developing an effective assessment program. Once there is agreement concerning what needs to be 
measured, the next question the self-study will face is "How?” The prevailing wisdom suggests that the institution 
should start with a few priorities and then measure them in several different ways using several different methods. 
In other words, it’s wise to use descriptive as well as evaluative measures (Palomba and Banta, 1999). 




Team learning is the fourth discipline; it is about synergy (the amplification of individual effects through interaction). 
When teams learn, not only are they producing extraordinary results, their members are growing and learning too. 
Team learning begins with a dialogue; people suspend judgment and agree to think together. It involves eliminating 
factors that impede learning. Competition, parochialism, and traditional academic rhetorical obfuscation all must be 
set aside in the interest of developing trust and common commitments. As Deming once suggested, such work “is 
not instant pudding” (Aguayo, 1990). 
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Tuckman’s [1965) classic model sets out four distinct stages of team development: forming, storming, norming, and 
performing. During the first stage, the team is formed; individuals are brought together and given a charter. If the team 
members are willing to extend themselves beyond perfunctory academic chitchat, they are likely to learn that their 
views and values vary. This is the beginning of the storming stage. It does not necessarily have to be acrimonious, 
but it is essential for the unique perspective of each member of the team to be recognized and the dimensions of 
difference between team members to be understood. Only after issues at this stage have been candidly explored can 
the team establish common values and aspirations. The high levels of common understanding and mutual trust that 
emerge as the norming stage concludes mark the beginning of the performing stage. 

Team learning is vital. Teams, not individuals, are the fundamental learning units in organizations; teams have the 
potential for much greater and more rapid learning than do even the brightest and most adroit individuals. 
Unfortunately academic teams [committees) are often the very antithesis of effectiveness. There is little difficulty in 
forming committees; they diffuse responsibility well and provide a convenient forum for interminable rhetorical 
obstruction. Commonly accepted faculty norms concerning committee work, once internalized in individual mental 
models, preclude authentic commitment, meaningful preparation, or necessary candor. As a result, most academic 
committees do not develop beyond the forming stage; they stagnate. 

Another mistake often occurs when well-intentioned administrators or faculty leaders try to short circuit the 
development cycle by skipping the storming stage and proceeding directly to the norming stage. Attempting to 
imprint a leader’s vision on committee members before they have had a chance to express their individual values 
virtually ensures that the committee will remain relatively immature, impotent, and ineffective. Such committees will 
find ways to generate the required "evidence” and a plausible narrative to support the leader’s particular perspective, 
but after all is said and done and the accreditation committee has left, nothing will have changed in the eyes or 
educational experience of the average student or faculty member. 

Systems thinking is the fifth discipline. All organizations are systems; each person and event within a system affects 
all other people and events in the same system. These patterns of influence are difficult to observe; most people focus 
on isolated parts and not on the system as a whole. Systems thinking encourages each person to pause and consider 
the potential causal connections that may be revealed by broader perspectives; it helps individuals and groups see 
how to effect desirable changes and collectively attain the outcomes they most desire. 

Success requires that both trust and understanding increase simultaneously. Understanding increases as established 
models are subjected to systematic inquiry. As an example, consider the popularity and assumed potency of 
collaborative learning. We in higher education believe that so powerful a pedagogy comes with a virtual guarantee 
of success. (There is a robust literature supporting the notion that team learning can lead to the attainment of a broad 
range of desirable outcomes.) Several years ago, while conducting a comprehensive assessment of the relative 
contributions of each of 35 general education courses to students’ knowledge, critical thinking, and intellectual 
curiosity at the Air Force Academy, we put this hypothesis to the test [Porter, 1998). To our surprise, the proportion 
of the course grade determined by group work was the only significant course characteristic related to any of our 
educational criteria. However, to our surprise [and dismay) the correlation was negative. In fact, among the nine 
courses in one particular academic division, the correlation was -.80. Collaboration can be a boon to teaching and 
learning; however when it is imposed on faculty and employed inappropriately, it can interact with other aspects of 
the system to produce unintended consequences or contrary results. There is simply no pedagogy so powerful that 
it can be applied without an understanding of the existing system. 

However, increased understanding of the process of teaching and learning cannot be achieved without also 
increasing trust, and the example cited above initially raised significant issues concerning trust and collegiality and 
the advisability of doing any further assessment. However, recognizing where the problem was localized [and how 
it came about), then marshalling the support necessary to fix the system, ultimately enhanced overall trust among 
faculty and administrators. Institutional leaders must realize that being trustworthy is a prerequisite to the 
development of trust among faculty. Basically this requires an institution to consider the alignment between its 
practices, policies, and espoused priorities. Misalignment among these components is likely to be most obvious to 
those farthest from the seat of power; this is why comprehensive, campus-wide surveys can be so valuable to those 
truly seeking to enhance institutional integrity. 

Simply being trustworthy is not sufficient to engender trust, however; the institution must also actively and repeatedly 
convey its trust of faculty and students. To do so requires it to seek out, then carefully consider, constituent perceptions 
of inconsistencies between its rhetoric and its practices. It must create a culture in which evidence is valued more 
highly than hierarchical authority. The challenge of leadership is to synchronize these activities and continually 
include more individuals and more constituencies in the institution’s most significant conversations. 
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In Summary: Five Modest Steps Toward More Successful Self-Studies 

Even the longest journeys begin with a single step, the proverb reminds us, so let us offer five beginning steps, related 
to Senge’s five disciplines that we believe will lead an institution toward authentic and successful self-studies. As a 
first step toward personal mastery, we must resist the natural urge to hurry the change process. It rarely works. We’ll 
save time and grief later if we plan for and take the time necessary to build trust, shared language, and a common 
commitment to our most important shared goals. This means, of course, starting the process well in advance of the 
reaccreditation visit. Second, to explore mental models, in addition to building trust, we can begin by sharing examples 
and evidence of our most successful programs and practices. Building on that second step, it will be easier to develop 
a shared vision of what is possible and to commit to its realization. Fourth, from the beginning, we can promote 
authentic team learning. This may require an expert facilitator to train us to work effectively together; it will inevitably 
require us each to develop thicker skins and more generous spirits. And fifth, we can apply systems thinking to our 
planning by asking how well ourvision fits within existing institutional structures and agendas, and what that suggests 
for change. 
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What Is a Mission Statement? 



Kristin M. Bowden 

Historically, whenever a person or group of people wanted to achieve something meaningful, they made a 
pronouncement of mission or purpose. One only has to look in the pages of the Bible, in great literature, in history, 
or even in society’s cultural avenues to find such pronouncements. For example, every episode of Star Trek began with 
these words of purpose: “Space, the final frontier. ... These are the voyages of the Starship Enterprise. Its five-year 
mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no man has gone 
before” (Abrahams, 1995). 

Mission statements are considered the starting points for most management programs, including TQM, self-directed 
work teams, management by objective, and divisional planning (Bart, 1997). The most important role of the mission 
statement, however, is to function as the foundation for planning (Pearce, 1 994). As a planning tool, it should serve 
to answer a variety of questions about the company (Pearce, 1994): 

o What is the organization’s self-concept? 
o What is the principal product? 
o What is the service area of the company? 
o Who are the primary customers? 
o Why do we exist? 
o What needs do we fulfill? 

The mission should serve management planning in a variety of ways. First, the mission should mirror top 
management's view of the strategic position of the company. It should also help cement established goals for the 
organization. Next, a mission statement should send a message to customers that the company has sound strategies. 
Fourth, it should inspire confidence in the organization. Finally, the mission should not only serve current planning 
needs, but also lay the groundwork for longer-range planning (Channon, 1997). James R. Lucas, president of Luman 
Consultants, feels that every company needs a clear mission or vision “to guide us, to remind us, to inspire us, to control 
us, and to free us” (1998, p. 23). 



Writing the Mission Statement 

The discussion to this point has assumed one basic fact: that the statements are well-written. While there is no one 
set way to go about writing a mission statement, several researchers agree on some basic elements that should be 
present. 

Professor John Pearce, endowed chair in strategic management at George Mason University, identifies several 
fundamental beliefs that should serve as prerequisites before an organization begins the process (1994, p. xi): 

o Belief that the organization’s product and service can provide benefits commensurate with its cost or price 

o Belief that the product or service can satisfy a customer need not currently met adequately for specific market 

segments 

o Belief that the technology to be used in the production process will provide a product or service that is cost 
and quality competitive 

o Belief that with hard work and with the support of others, the organization can do better than just survive; it 
can grow and meet financial objectives 
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o Belief that the manager’s and leader’s philosophy will result in a favorable public image and will provide 
financial and psychological rewards for those willing to invest their labor and money 

o Belief that the self-concept that the managers and leaders have of the organization can be communicated 
to and adopted by employees, investors, and other stakeholders 

o Belief that the organization can provide a quality product or service that satisfies a customer need. 

If an organization can buy in to these belief statements or a similarset of its own making, it is ready to begin the arduous 
task of developing a mission statement. 

Many researchers agree that four basic elements or parameters comprise a mission statement: audience, length, tone, 
and format or approach (Abrahams, 1995; Drohan, 1999). Who is the statement written for-employees, customers, 
stockholders, etc., or any combination? Knowing the target audience will aid in answering other important questions. 
How long should the statement be? Some companies have long statements, and others have single-sentence 
statements. The fundamental guideline, however, is for the mission to “be long enough to reach the target audience” 
(Abrahams, 1995, p. 47). 

Tone will be different for each organization. Again, make sure that it is written in the language of the target audience. 
"Lofty or ponderous” statements should be avoided (Drohan, 1 999). Jeffery Abrahams asserts that certain keywords 
will also help set the tone. In his survey of more than 300 mission statements, he found that many words were used 
numerous times and helped the audience understand the tone of the statement. The top ten words used are listed, 
followed by the number of times they were cited: service (230), mission (221), customers (211), value (1 83), employees 
(157), growth (118), environment (117), shareholders (114), success (105), and leader (104) (1995, pp. 49-50). 

An organization must carefully choose the words to be included in its mission. For example, if a word such as profit 
is used consistently, the statement will take on a different meaning than if the word customer is used. 

The fourth element in the mission is the format or approach of the statement. In other words, how is it presented to 
the audience? The approach should be determined by the culture of the company. Some companies formally print 
the statement on card stock and display it; other organizations laminate a small card that can be carried in a wallet; 
and some print the statement on the back of business cards. Regardless of the method, the most important aspect 
is to get the statement to the audience (Abrahams, 1995). 

After the parameters of audience, length, tone, and format have been set, the next step is to decide who is going to 
write the mission statement. Top management must give full support to the process. Support can mean either specific 
language to be included or general ideas to be represented. Either way, management must be involved in every step 
of the process (Pearce, 1 994). Creation of a committee to review or write the mission statement would seem to be the 
most logical step. Inclusion on the committee must not be limited to management or even to employees. “Involvement 
of middle managers, shareholders, and customers... was found to have a more positive impact on ... outcomes. 

The mission development process should not be an autocratic or top-down exercise” (Bart, 1999, p. 37). In other 
words, give the people who will be living the mission a hand in writing the mission. 

Once the decision is made about who will be involved in writing the mission statement, the organization can begin 
the writing process. John Pearce outlines five steps he sees as important in writing the mission statement (1 994). First, 
the organization should gather as much information about the company and its goals as possible. Second, those 
involved should produce a rough outline of the proposed statement. The writers should then get as much feedback 
as possible from all levels of the organization. At this point, the mission statement should be revised. The final step 
outlined by Pearce is to get management approval of the final product. 

While these steps may seem simplistic, they provide a framework by which to accomplish the set goal. When gathering 
information about the organization, it is very important to consider what details in a mission statement can be 
associated with high performance. According to Christopher K. Bart, these items include (1998, p. 56): 

o A statement of purpose or general, nonfinancial goals 
o A statement of values 
o Specification of behavioral standards 
o Identification of the organization’s competitive strategy 
o A statement of vision-big, bold, and long-term 

o An expression of intent to satisfy the needs and expectations of the stakeholders. 
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Other researchers also believe that it is imperative for the mission statements to include a clear picture of the 
organization and its product [Newsome & Hayes, 1990). In a study of 90 public higher education institutions, however, 
researchers found that a small percentage of postsecondary institutions included all of the necessary dimensions: 
only 30 percent contained target audience, 35 percent contained a self-definition, and most troublesome, only 
33 percent defined their product [Newsome & Hayes, 1990). 

After the mission is complete, many organizations feel that the work is complete as well. In reality, the most difficult 
task is just beginning. The statements must now come to life. “Deeds must back up the rhetoric" (Pearce, 1994, p. 7). 
Dissemination is very important, along with review processes, in aiding the employees’ understanding of the vitality 
of the mission. In fact, the North Central Association of Colleges and Schools, an accrediting agency that outlines the 
importance of mission to higher education institutions, essentially begins with an appropriate mission and then looks 
for patterns of evidence that the mission is in use. Such patterns include (NCA, 1995): 

o Long- and short-range goals are in place 
o Constituencies are involved in evaluation of mission and purposes 
o Decision-making processes are appropriate to the mission 
o Constituencies understand the mission and purposes 
o The public is kept informed about purposes. 

Perhaps the most important question can now be asked: Is the mission statement making a difference to the 
organization? 
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Organizing the Self-Study for 
Yourself, the Steering Committee, 
and the Campus Community 



Clark Jenkins 



Conducting an institutional self-study and preparing the self-study document require a tremendous amount of time 
and effort. As Self-Study Coordinator, you may feel like you have all of the responsibility and none of the authority 
to get the job done. To a large extent, this is true. The only way to persuade colleagues to help with what many will 
view as your task is to convince them that (1) it is in their best interest to help, [2) you know what you are doing, and 
(3) you are well organized, so you will not be wasting their time. Efficient organization is the only way you're going 
to pull off the first two items, and as for the third-well, you’ve got too much to do to waste any of your own time, either. 

The following statements highlight some of the key aspects of organizational strategy that worked for us at North 
Arkansas College. In order to provide clear guidelines to everyone involved, the Steering Committee published 
campus-wide a Guide for Conducting the Self-Study prior to organizing the subcommittees. The Guide contained 
detailed instructions regarding responsibilities of participants, what should and should not be done when collecting 
and analyzing data, and a style guide for writing the self-study document. It also included copies of the Criteria for 
Accreditation and GIRs. 



Start Early 

At North Arkansas College, the self-study co-coordinators were appointed, one from each of our two campuses, 
during the Fall, 1997 semester for a comprehensive evaluation visit scheduled for the 2000-2001 academic year. This 
was not too much time! 

If possible, consider sharing your responsibility with a co-coordinator-1 can’t imagine trying to do this alone-and 
decide on a division of labor. My co-coordinator and I decided very early that he would be responsible for the resource 
room and would handle all requests for information. I was responsible for basic organization and for keeping everyone 
on schedule. He was the main editor of the self-study document, while I wrote the introductory and concluding 
chapters. Make sure you get release time. I was given release from one class per semesterfor the duration of the self- 
study process. My co-coordinator, a librarian, was given the equivalent release from his duties. 



Do Your Homework Before You Select a Steering Committee 




Study the NCA Handbook of Accreditation like it is your only text from which to prepare for a comprehensive exam. 
Notice that there are five Criteria for Accreditation and 24 General Institutional Requirements [GIRs) that your 
institution must meet to be granted continued accreditation. The Handbook also cross-references the Criteria for 
Accreditation and the GIRs. Rough out a tentative outline of how you might arrange these as topics in the self-study 
document. Think of the individuals on your campus with a reputation for getting things done, and think of how their 
interests might match the rough outline you have prepared. Talk to these people. Share your outline with them, ask 
for their feedback, and modify the outline in response to their suggestions. Ask if they are willing to serve in some 
sort of leadership capacity. You are not promising Steering Committee appointments at this time. What you are doing 
is interviewing candidates for your top management team (Steering Committee) and using their feedback to polish 
your outline. 
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Keep the Steering Committee as small as possible, and don’t be afraid to delegate authority. One of the mistakes that 
was made while preparing for our 1990 comprehensive visit was having a Steering Committee that was too large. By 
the time everyone got there, settled down, etc., it took forever to conduct business (I was one of the members]. This 
time, we had two coordinators and one Steering Committee member for each of the five Criteria for Accreditation. 
Each Steering Committee member oversaw from one to four subcommittees. Each subcommittee was responsible for 
writing one chapter of the self-study. 



Attend the NCA Annual Meetings 

Try to get a budget that allows as many people on your Steering Committee as possible to come to the NCA Annual 
Meeting every year during the self-study process. Attending the Annual Meetings is the single best tool available for 
getting the Steering Committee focused on its job and for getting a firm outline of the self-study in place. 



Da Doolicfi r* 



At the outset, the Steering Committee had all sorts of ideas about howto build interest in the self-study, howto keep 
the campus community informed, and how to persuade people to stay on schedule. We were going to have our own 
page on the college web site (updated weekly], contests (complete with prizes, of course, for those who finished first], 
and so on. None of this happened. Why? Because it all took time, and none of us had time for something that didn’t 
really have to be done. Memos (both email and snail mail] were faster, and we didn’t need the contests. Stick to the 
basics. 



Establish a Timeline 



The Handbook of Accreditation has a suggested timeline that is an excellent starting point for organizing your time. 
Remember to submit your institution’s timeline to your NCA staff member for feedback. This paper does not have room 
for a detailed timeline, but in general terms, here’s how it went for us: 



Fall 1997: Coordinators get organized. 

Spring 1998: Steering Committee is selected and gets organized. 

Fall 1998: Subcommittees are selected and organized; resource room is started, and institution- 

wide search is initiated for copies of all reports, statistics, etc., that might be useful in the 
self-study; all materials are cataloged and indexed as they come into the resource room. 



Spring 1999: Subcommittees compare their data needs with what is already available; a master list of 

"unknown data” is compiled so that one campus-wide survey can be conducted to 
collect all missing information; preliminary outlines of each chapter are presented to the 
Steering Committee. 

Fall 1999: Campus-wide self-study survey is administered, and first drafts of all self-study chapters 

are completed. 



Spring 2000: First draft report is circulated campus-wide to check for completeness and correctness; 

second draft is circulated to ensure that all needed corrections were made. 



Summer 2000: Final draft is printed; Basic Institutional Data Forms are completed; Third Party Comment 

is sought; all necessary materials are sent to NCA and evaluation team members. 



September 18-20, 2000: Successful Comprehensive Evaluation Visit! 



Use Only Volunteer Labor 

You are the only one who will get any release time for working on the self-study (and whatever you get won’t be 
enough]. For everyone else, this is going to be one more committee assignment on top of an already bulging workload. 
During the Fall 1998 pre-semester workshops, our NCA staff member was on hand to help us make our case to the 
f aculty and staff. We asked for volunteers to indicate their first, second, and third choices for areas in which to serve. 

315 



Chapter 14. New Designs in Higher Education. Coordinating the Self-Study/ 323 



More than half of the college’s full-time employees (96 out of 167 faculty and staff) volunteered. The Steering 
Committee compared their individual strengths and weaknesses, who they did and did not get along with, and 
whether both campuses and all areas on each campus were getting uniform representation on each subcommittee, 
and we actually managed to place everyone in one of their chosen areas. It was important to make sure that each 
subcommittee had at least one good organizer, one good analyzer, one good leader, and one good writer. As we saw 
where mistakes had been made, we quietly shuffled a few individuals from one subcommittee to another and allowed 
a few people to drop out. Even with strong, open support from the president and the rest of the administration, 
remember that everything you get you have to ask for; you are not in a position to tell anyone to do anything. It is 
occasionally necessary to remind subcommittee members of this, as well as yourself and the rest of the Steering 
Committee. 



Don't Have Unnecessary Meetings 

Don’t insist on regular meetings whether there is a reason to meet or not. Our Steering Committee met weekly during 
the planning stages, almost not at all during the data collecting and writing phase, and at several all-day sessions 
putting together the first and second drafts. Between meetings, business was handled informally via email, telephone, 
and personal contact. The subcommittees conducted business the same way. We insisted that everyone-including 
ourselves-keep and turn in some sort of log of all actions taken outside of formal meetings fas well as minutes of 
all meetings). 



Create Clear Job Descriptions 

Our Guide for Conducting the Self-Study listed the responsibilities of all participants, including Self-Study 
Coordinators, steering committee members, subcommittee chairs, and individual committee members. It outlined the 
responsibilities and structure of each subcommittee and included the topics that each subcommittee was expected 
to address. We made it clear from the start that such detailed instructions were not intended to be dictatorial, but rather 
to be thorough. By listing detailed descriptions of everyone’s job in a publication given to everyone, it was hoped that 
any detail we had overlooked would be brought to our attention (it was, fortunately, on several occasions). 



Provide a Style Guide 

It was our intention from the start that each subcommittee would present to us a completed chapter as close as 
possible in appearance to the way it would look in the final self-study document. We were determined that, except 
for the introductory and concluding chapters, we would be editors, not writers. This required a detailed set of 
instructions dealing with everything from use of third person and other grammatical considerations, to how to handle 
our decision to not use appendices or footnotes, to keeping in mind the multiple audiences for whom the report was 
being written. It also included instructions on fonts, spacing, howto present charts and graphs, what word processing 
program to use, and how to submit documents. As a result, minimal editing was required to get the self-study to read 
as one complete document rather than a collection of disjointed reports. 



Say "Thank You" and "Congratulations" 

After the evaluation team has left, after their final report has arrived and been disseminated institution-wide, you need 
something to say “thank you” to every member of the college community who helped with the self-study. You also 
deserve congratulations for a job well-done. We took care of both by planning a party. Actually, we asked our President 
to plan a party. He did. It was wonderful. We all deserved it! 



Clark Jenkins is instructor, Physical Sciences, at North Arkansas College in Harrison. 
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Suggestions on 
How to Make Your 
Mission Impossible Turn into 
Mission Accomplished! 



Kathy Burlingame 



Your mission, should you choose to accept it, is to help your college effectively complete a self-study process, produce 
a Self-Study Report, and conduct a successful on-site visit. 

For most of you reading this article, the responsibility to coordinate a self-study process has already been delegated 
and accepted. As a new Self-Study Coordinator, you may be experiencing emotions on a continuum ranging from 
excitement and anticipation due to new challenges all the way to some anxiety or concern [OK, fear) as you have been 
chosen to leap out of a moving institutional life into a new and unknown educational frontier. If John Wayne were 
to show up with Tom Cruise in your institution’s next Mission Impossible f\\m, John’s character might encourage Tom’s 
character by saying, “Cheer up, Pilgrim, this is a doable task, and we can accomplish this together!" Enclosed are some 
ideas to help you get started. 



Research 

0 Read the NCA HandbookUom coverto cover! I highlighted, wrote questions, and made notations all over mine. 

I found it to be a very helpful resource. 

0 Review previous documents from your institution’s laston-site visit, starting with the most recentTeam Report, 
including the institutional strengths and recommendations. 

0 Gather and scan documents from other institutions that have recently gone through the accreditation process. 
Include a search on the Internet for specific topics such as assessment, mission, and institutional integrity. 
Initially, I started with a broad search of the topics and did not limit the search by institutional levels. I am glad 
that I did not focus only on combined community and technical colleges, as I would have missed some of the 
rich resources and ideas presented by larger four-year colleges, private schools, and stand-alone community 
or technical colleges. Scan NCA’s web site to familiarize yourself with their resources. Review previous articles 
published in recent editions of NCA’s A Collection of Papers on Self-Study and Institutional Improvement 

0 Find out what institutions in your area are working on the self-study process. (Your NCA staff liaison may be 
an excellent source for this information.) Look for one or two institutions whose on-site visit is at least 4-6 
months ahead of your scheduled visit. Ask to observe one of their Steering Committee meetings or planning 
sessions. Thankfully, several northern Minnesota colleges were scheduled for accreditation visits a few 
months prior to Northland’s on-site date, and they allowed me to share in some of their meetings. Their Self- 
Study Coordinators and campus Assessment Chairpersons met to discuss NCA’s General Institutional 
Requirements, models of approach for the self-study process, assessment, and other issues. Also, this group 
hosted a meeting with their NCA staff liaison to address specific questions. Having a group meeting was an 
effective use of the liaison’s time, and it helped to hear other people's questions! Finally, ask to visit at least 
one institution’s Resource Room prior to their on-site visit. 
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Mentoring and Support 

0 Initiate communication with your institution’s NCA liaison. Connecting with a real person who can help you 
recharge your emotional batteries, rekindle the vision, and give you direction if you get overwhelmed is 
essential. 

0 Seek out a mentor or a knowledgeable, objective person external to your institution to act as a sounding board. 
Look for someone who can encourage you in the self-study process and is willing to share his or her wisdom, 
ideas, and sample documents. An external perspective is healthy in that it encourages you to think out of the 
box of your institutional culture. Also, an external mentor provides you with an objective resource who may 
energize the self-study process by asking tough questions that may need to be considered while completing 
the self-study process. 

0 My mentor was very helpful in getting me started with the self-study process. She showed me how she was 
organizing the data as they were coming in and shared with me her approach to writing the report. For 
example, some of the historical information of what the college has done since its last on-site visit can be 
written earlier in the self-study process. Also, she encouraged me to document the General Institutional 
Requirements early in the process, as these are foundational to the Self-Study Report. Finally, my mentor was 
a contact who eventually opened the doors for me to attend some of the regional meetings for Self-Study 
Coordinators and Assessment Chairpersons. 



Steering Committee 

The establishment of a strong Steering Committee is essential to the self-study process. It is helpful to identify specific 
expectations and qualifications for membership in the Steering Committee prior to choosing or soliciting volunteers. 
Keep in mind that is important to have broad representation from all constituent groups in the self-study process and 
in the Steering Committee membership. Northland Community and Technical College chose to follow the fairly 
common approach of having each of the Steering Committee members serve as a co-chair of one of the 
subcommittees. A brief list of some of the qualifications for Steering Committee membership is included to stimulate 
discussion on your campus. Steering Committee members should: 

o be objective-be able to view a situation from more than one perspective; 
o have a history of effective problem solving and critical thinking skills; 
o possess leadership skills and be a team player; 
o have a can-do attitude and do the work that is expected; 

o demonstrate effective communication skills, be a good listener, and be respected by peers; and 
o be employees whose scope of influence includes the expertise necessary to achieve the institution’s goals. 

During the time of Northland’s self-study process, the college experienced a change in administration in the position 
of the Dean of Academic Affairs, and two co-chairpersons retired. Both subcommittees completed their assigned 
tasks under the leadership of the remaining co-chairpersons, who had a can-do attitude! 



Model or Institutional Approach to the Self-Study Process 

Identify the type of approach or model that fits well with your institution. Four different models are presented here 
as examples to start your creative juices flowing. The best approach is the one that fits your institutional culture and 
is consistent with your institutional goals relative to the self-study process. 

0 Some institutions approach the self-study process purely from the perspective of the five criteria. Subcommit- 
tees are created and charged with fully documenting an institutional response to each of the examples of 
patterns of evidence listed under each criterion. 

0 A functional approach toward the self-study process requires a little more vision and planning at the start of 
the self-study process but may yield a rich and holistic perspective of the institution. For example, each functional 
area of the institution is charged with documenting patterns of evidence relative to their service or functional 
area [e.g., assessment student services, academic or instructional, facilities and physical plant, and so on). 
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0 Another interesting model is to view the institution from a continuous quality approach and identify priority 
areas of concern or focus. One institution used this model and identified two major themes as the foundation 
for its self-study process [improving student services and communication). Each of the criteria was addressed 
with an emphasis on the patterns of evidence relative to these two areas. 

0 Connecting the institution's strategic planning efforts to the five criteria and the self-study process is another 
model that has been successfully utilized. 



Northland Community and Technical College's Approach to the Self-Study Process 



Northland Community and Technical College [NCTC) used the functional area approach for its self-study process by 
evaluating the criteria from the perspective of the major operating areas of the college. The major operating arfeas 
or functional areas of the college help define the frame of reference for each the subcommittees. Each subcommittee 
evaluated the college from the framework or perspective of their functional area. Thus, the employees serving on the 
various functional subcommittees of the college looked at each criterion from diverse points of view. The college’s 



0 Northland Community College and Northwest Technical College merged into Northland Community and 
Technical College in 1995. Historically, the two colleges co-existed as separate institutions with their own 
specialized missions, philosophies, and purposes. Although the two colleges became one institution on paper, 
the employees of the college were still evolving in a cultural transition to change perceptions, organizational 
climate, and language in order to become one college. At the beginning of the self-study process, barriers 
existed that were impeding employee dialogue and work. 

0 The functional model did positively assist in changing the college culture. Also, this approach encouraged a 
fairly high level of participation in the self-study from all constituents and did result in a holistic evaluation of 
the college. 

0 The process of employees working together, without regard to union membership or department designation, 
for the purpose of accomplishing a self-study did promote further dialogue and ah increased awareness of 
the unique strengths and qualities that each employee and program area brings to the college. Also, this model 
encouraged employees to get to know one another and to learn from each other. Increased communication 
and interaction among employees helped to further develop an institutional culture that ultimately strength- 
ened the college. 



Develop a "Mission Possible" Timeline 

Start with the draft timeline presented in the NCA Handbook as an initial framework. Consider other institutional 
events and priorities when planning your timeline. What will work on your campus? Do you have blocks of time and 
dates set aside to accomplish institutional tasks? Talk to publishing or printing companies now to determine 
necessary time frames for printing the Self-Study Report cover and final document. The discussion and evaluation 
of philosophical issues, such as the review of the mission and vision of the institution, should occur early in the self- 
study process. The evaluation and possible revision of philosophical statements need to be well-thought-out, and 
there Should be a certain amount of “gel” time to allow people to live with the new ideas or proposed revisions. Due 
to Northland’s recent consolidation, the discussions abolit philosophical issues were very important to the future of 
the college and the self-study process. 



Identify Subcommittees 

Orice a model or method of approach is chosen, it is easier to identify Subcommittees. Existing campus committees 
may be accessed as a valuable resource. Consider whetherthe existing campus committees have a broad constituent 
representation. Also, it may be helpful to establish an ad hoc subcommittee composed of some of the members of 
established campus committees as well as new members. Remember, one of the purposes of the self-study process 
is to be reflective and evaluative. Having “new” people come to a group provides the impetus for looking at policies, 
procedures, and processes from different perspectives. Although it may seem painfully obvious to the members of 
an established committee how the patterns of evidence are met, it may be beneficial to have “outsiders” collaborate 
in the evaluation of the patterns of evidence. Also, having students on the subcommittees brings a unique and rich 
perspective to these discussions. 
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Howto Deal with Cultural Quagmires, Emotional Bogs, and Other Cliffhangers! 

As is the case with most institutions today, Northland Community and Technical College was recovering from a 
tremendous amount of change and turmoil at the time of our self-study. I will not bore you with long saga of events- 
but believe me, NCTC has gone through and survived a lot! As a result of honest self-reflection, good planning, and 
commitment from the employees, I believe that our college is stronger and healthier for having completed our self- 
study process. Often, institutional cultures change slowly and require active change agents to facilitate change. Small, 
seemingly subtle changes may make a big difference. 

o For example, we started using the term employees wherever possible instead of faculty, staff, and administra- 
tion. This was a healthy cultural decision in an environment that was somewhat divided by union categories. 

o Acknowledge past perceived injustices or hot issues, but limit the lamenting about the good old days. 
Capitalize upon what was good, and make logical connections to where the institution currently is. 

o Recognize that strong emotions about issues are better than indifference! Look for the root of the strong 
emotions, and listen for the real messages being sent Encouraging subcommittee members to write down 
the issues may help defuse volatile situations. 

o Being a Self-Study Coordinator may make you privy to confidential or sensitive information that may not have 
direct implications to the Self-Study Report. If in doubt, clarify what may or may not be shared with others. 

o Listen to all sides of an issue, and choose your wording and timing wisely in trying to present another group’s 
perspective on hot issues. 

o Recognize that the self-study process is bigger than one person and that the Self-Study Coordinator does not 
have to have all of the answers! 

o The self-study process is more of an institutional journey than a final destination. Keeping focused on the 
process will ensuhe an excellent product. 



Writing the Self-Study Report 

It is important to recognize that the Self-Study Report will present a snapshot of the institution at a specific time and 
juncture. It is reasonable that some things may change from the time of printing the Self-Study Report to the time 
of the on-site visit. These changes can be communicated to the on-site team of Consultant-Evaluators through a 
memo or other forms of appropriate documentation. There comes a time when you have to say, “It is finished, and 
it is as representative of our institution as possible” and send it to the printer. Other key points are as follows: 

o Make sure that the Self-Study Report is more evaluative than descriptive. Obviously, there needs to be enough 

description in the report for intended audiences to understand the culture and unique characteristics of the 
institution, but the purpose of the self-study process is to be reflective and evaluative of your specific 
institution. 

o How honest should you be? Very honest! The Self-Study Report can be honest and sincere while still 
presenting the issues in a “cup-half-ful!” instead of a “cup-half-empty” perspective. One of the most insightful 
experiences that reinforced to me that we needed to be very honest about out institutional issues occurred 
as a result of hiring an outside consultant to help the college write an enrollment management plan. The 
consultant was on ourcampusforoneand a half days and identified almost every key issue or area of concern 
that we had identified through several months of self-evaluation and reflection! Now if one good consultant 
could see all of this in one and a half days, how much more would several Consultant-Evaluators see in three 
days? The purpose of the Self-Study Report is not to air all of the institution’s dirty laundry but to honestly, 
sincerely evaluate the institution’s strengths in order to capitalize on them and to identify issues that need to 
be improved upon. In addition to identifying these issues, it is imperative forthe Self-Study Report to document 
realistic plans to help the institution move forward in the achievement of its goals. Acknowledging successes 
and failures and exposing institutional strengths and weaknesses requires courage and integrity. This process 
is the opportune catalyst for change. The more honestly an institution embraces its own issues, the more likely 
it is that the institution will continue to grow and change for the better. 

o Utilize the judgment and expertise of the Steering Committee to help determine what should and should not 
be in the Self-Study Report This is helpful in situations when individuals have an ax to grind and their input 
does not reflect the general consensus of the group at large. 
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o Don’t waste a lot of time and effort duplicating information that is in other documents, such as the institution’s 

catalog, student and employee handbooks, contracts, and so on. Evaluative comments about information in 
these documents are helpful, and specific quotes may be beneficial, but don’t cut and paste whole published 
documents into the Self-Study Report. 

o Determine how often drafts of the various sections and the final Self-Study Report will be distributed to all 
employees. 

o Let the final draft of the Self-Study Report sit for a week or two. Go back and reread the whole document with 

a fresh mind for the last time before sending it to the printer. 



Resource Room 

Start collecting documents and pertinent information early! I would recommend setting up the Resource Room based 
upon the Criteria. Use open tables, three-ring binders, and magazine files to display the resources. Keep a running 
list of documents added to the Resource Room to create a final indexed list of the available resources. Have chairs 
spread around the room so that team members can comfortably read the documents. Decide early on a template for 
covers and spines to be inserted in the three-ring binders. Request that chairpersons and department heads start 
compiling their resources early and put them into logical categories. 

Tips on Preparing for the On-Site Visit 

o Consider conducting a mock on-site visit prior to the real one. This was an extremely valuable experience for 
Northland Community and Technical College! 

o Be specific about the qualifications or characteristics your institution is looking for in the Consultant- 
Evaluators. For example, we knew that we needed to improve on assessment, and we asked NCA to include 
someone with expertise in this area. 

o Remember that the Consultant-Evaluators wear two hats. They will evaluate the Self-Study Report and other 
documents for accuracy to ensure that the Criteria for Accreditation and General Institutional Requirements 
are addressed. In addition to this evaluative role, the team also functions in the consultant role. Consultant- 
Evaluators are in a position to really help the institution achieve its goals through their input. They are real 
people invested in helping the institution while meeting NCA’s delegated functions. Our Consultant- 
Evaluators were excellent, kind, and very helpful to the institution! 

o Collaborate with the Team Chairperson to establish an agenda for the on-site visit. Be prepared to discuss 
lodging and other travel arrangements. Ask about special dietary needs or other necessary accommodations. 

o On the days of the visit, have people assigned to escort team members to the appropriate meeting places. 

o Creature comforts are appreciated. Ask what kinds of beverages are preferred. In addition to meals, have 
portable snacks available, such as fruit, nuts, mints, and something for the sweet tooth. The team members 
put in long days and appreciate thoughtful touches. Don’t go overboard, but make them comfortable. 



Specific Strategies Northland Used to Increase Student and Employee Awareness Prior to the Visit 

o We placed tent cards with pertinent information on tables in areas such as the cafeteria or lounges. This raised 
students’ awareness about the impending on-site visit and explained its purpose. 

o Employees participated in a button design logo contest. We had the winning idea made into pin-on buttons 
and distributed the buttons to employees and student leaders. 

o Other creative ideas generated from the button design logo contest were used as part of some wonderful 
bulletin board displays. Also, employees and students used themes related to accreditation and one of the 
criteria on bulletin boards in areas such as the Learning Center and the library. 

o We utilized our internal communication systems to provide timely information to employees. For example, brief 
summaries were presented in our college employee and student weekly newsletters. The college newspaper 
published articles related to the on-site visit. 
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o The college's mission and vision statements were placed on the intercampus electronic bulletin system and 
on a large-screen TV in the commons area. The computer operations center technicians magically placed new 
screen savers on the computer lab monitors that reminded users about NCA. 

o A series of one-page informational flyers were emailed to all employees and posted around campus during 
the final days just prior to the on-site visit. 



Conclusions 

As a first time Self-Study Coordinator, I was professionally challenged by this task. I was given the opportunity to move 
out of my comfort zone and to broaden my knowledge base in numerous areas. One of the most satisfying personal 
and professional outcomes that I realized was the opportunity to network with so many wonderfully creative and 
innovative professionals within and external to our institution. The mission of completing a self-study process is 
impossible for one individual to accomplish. However, as your constituents collaborate to accomplish institutional 
goals and achieve milestone after milestone by completing each step in your institutional timeline, you will one day 
soon be able to say, “Mission Accomplished!” 



Kathy Burlingame is a Nursing Faculty at Northland Community and Technical College in Thief River Falls, Minnesota. 
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Understanding Your 
Campus Culture Is Key to 
Self-Study Success 



Gloria Dohman 
Harvey Link 



The self-study process appears to be overwhelming at first, but with careful planning and attention to detail, it can be 
both manageable and rewarding. Throughout the years, the North Dakota State College of Science (NDSCS] has found 
that engaging in a purposeful planning process is central to the success of a project of the magnitude of an NCA self- 
study. 

Becoming familiar with the NCA expectations and requirements is a critical first step in this process, but nearly as 
important is knowing your institution’s organizational structure and culture. Without a clear understanding of these 
two factors, it will be very difficult to successfully complete a quality self-study. It is the purpose of this paperto explore 
the planning approach that was used at NDSCS during our recent self-study, and to share some of the rationale 
behind the various steps. 

First, it must be made clear that there is no “right way” or “best way” to complete a self-study. So much depends on 
the organization and the culture that permeates it. The purpose of this discussion is to help institutions realize the 
need to design a process that is right for their own organization. With a successful planning process, the 
implementation of the plan becomes much easier. 



Familiarize Leadership 

Who will lead the self-study process? For that matter, who will have the ultimate responsibility for assuring that the 
institution is successful in meeting the accreditation standards set forth by NCA? It may be a director or coordinator 
of assessment, a vice president or dean of instruction, or even the president. Whoever these individuals are, it is critical 
that very early in the process they become intimately familiar with the expectations and requirements of NCA related 
to accreditation. Attending NCA Annual Meetings and accreditation workshops, visiting with colleagues who have 
recently completed the accreditation process, and meeting with the institution’s NCA staff liaison are just a few ways 
to start. But don’t wait to get started; it will take more than one meeting to get all of the information you will need. 

In the early stages of planning, it is especially important that the senior leadership of the institution support and 
demonstrate knowledge of the accreditation expectations and requirements. It is critical to send a clear message to 
the college about the importance of the endeavor as well as the magnitude of the project at hand. 

At NDSCS the familiarization process began well before the inception of the NCA self-study. The institution was 
fortunate that the college president was serving as a NCA Consultant-Evaluator and had an excellent working 
knowledge of the goals and objectives of the process. The college had also actively participated in NCA Annual 
Meetings by having one or more individuals attend forthe preceding several years, and began to strategically increase 
the number and makeup of individuals attending in the years leading up to the self-study. Conversations with the 
college’s NCA liaison proved very helpful in providing insights that matched the needs of the campus. All of this proved 
^ helpful to the process. 

o 
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Designate Lead Personnel 

Who will lead and coordinate the process on a daily basis? Who will write the Self-Study Report, and what about an 
editor? These are probably not your senior administrators, as they will not be able to dedicate the amount of time and 
effort to do the task effectively. However, it is important to identify at an early stage the personnel who will lead the 
effort on a daily basis and to allot adequate time for these responsibilities. Whoever will be charged with guiding the 
self-study must have enough time to become the resident expert in accreditation and needs to be intimately involved 
in determining how the self-study will be conducted. 

In preparation for the reaccreditation process at NDSCS, the college established the position of Director of 
Assessment and Institutional Research. This position was filled by an individual who was familiar with the campus 
and knowledgeable about assessment and accreditation. The establishment of a full-time position for this purpose 
was central in demonstrating the importance the college was placing on these initiatives. Most importantly, it gave 
a focus and qualified leadership to the effort. This individual provided daily guidance and coordination of the process 
and also became the primary author of the self-study document. 



Evaluate Campus Culture 

What type of organizational culture exists on your campus? Are employees empowered and encouraged to function 
in a primarily self-directed manner, or is the decision making process centralized, with the majority of the leadership 
coming from a handful of individuals? What about the committee and employee work-group structure; does itfunction 
smoothly and effectively? Whatever the structure or organizational culture that is in place at the time, it is critical that 
the self-study process complement and draw on its strengths. Now is not the time to introduce a major deviation in 
how your organization functions! 

At NDSCS the faculty and staff had been involved with a major total quality initiative for several years, so it was natural 
for a very decentralized and highly interactive process to be established. In fact, it was at this point that the theme 
of the study, “A Spirit of Continuous Improvement,” began to emerge. 



Develop Strategy 

What type of self-study strategy will effectively use the structure and culture of the campus to meet the NCA 
expectations and requirements? Who develops the blueprint for action? What processes are in place that can be built 
on? How will all of this be communicated to the greater campus? What are the timelines? Who is responsible for what? 
Will the process meet with NCA’s approval? 

At this point, the true planning process really takes place. What happens here will guide the rest of the activities, and 
it is critical that this step not be shortchanged in either time or effort. It may be designed by a relatively small group, 
but it is imperative for those taking part to be very familiar with the NCA requirements and accreditation process as 
well as to have an excellent working knowledge of the campus. 



At NDSCS it was the goal to integrate the self-study into the daily routine of the campus and to demonstrate that the 
processes used to analyze and evaluate were continuous and sustainable once the team site visit was completed. The 
challenge was to gather and organize the information in a manner that addressed the expectations of NCA for the 
Self-Study Report, especially as it related to how the GIRs and criteria were being met. To do this, a thoughtful review 
of what structures were already in place and what would need to be added in order to accomplish a meaningful self- 
study was necessary. Following this, a process was established that complemented the campus culture and 
organizational structure. It also reinforced the importance of data collection for continuous improvement. 




A process for selecting steering committee members, finalizing the plan, establishing timelines, communicating with 
the campus, initiating and conducting the study, writing the report, and other related steps fell into place once the 
initial planning activities were completed. Because NDSCS had been operating under a philosophy of continuous 
improvement for a number of years, there was a wealth of information collected that could be incorporated into the 
self-study and become the basis for analysis and evaluation. Therefore, it was hoped the culture and structure of the 
institution would become the framework for the self-study. 
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Secure "Buy-In" 

Will the planned process work? Is it compatible with the campus organization? Has anything been forgotten? Will it 
be supported by formal and informal campus leadership groups? Are the timelines realistic? Have adequate resources 
been allocated? How will the plan or process be communicated to the rest of the campus? 

Critical to the success of the self-study process is wide involvement of all constituencies on the campus: faculty, staff, 
and students. Until now, the proposed process has probably been developed by a reasonably small group of 
individuals. The need to share it with an ever greater number of influential campus members is critical. Meetings to 
seek the input from key academic administrators, faculty leaders, students, and prospective steering committee 
members is an important step in securing their buy-in. It is also a means of gathering valuable feedback and 
suggestions for improvements. This feedback should be taken very seriously, as it will form the basis of support for 
the process. 



At NDSCS this process began with both formal and informal discussions taking place between those developing the 
process and other key academic leaders. Frequent conversations and updates at relevant committee meetings proved 
helpful in beginning to acquaint the campus with the proposed process. It also helped solicit feedback about potential 
modifications that could improve the process. This also helped identify strong candidates to serve on the steering 
committee. 



Finalize Plan and Timelines 

Is the process realistic and doable? What will NCA think of the plan? Do the timelines make sense? Once the plan 
is designed, it needs to be forwarded to the institution’s NCA staff liaison for review and approval. It is the goal of NCA 
that each institution have a meaningful and successful self-study, so it is important that the plan be workable and 
realistic in nature. The NCA liaison is a valuable resource whom each institution can draw on. In addition, this step 
provides a point at which both NCA and the institution can measure whether the project is on track for successful 
completion. 

At NDSCS the guidelines provided in the NCA Handbook proved very useful in establishing preliminary timelines for 
the project. Anticipated assignments were made and discussed with those who would be responsible for them. The 
plan was finalized and approved by the College Council and submitted to the institution’s NCA liaison. Regular contact 
with the campus liaison proved very helpful during this time. 



Implement Strategy 

Now the real work will begin. The blueprint for action has been established, and it is time to implement it. A procedure 
to check the progress of the project is needed to be certain that all the actions remain consistent with the plan. The 
Plan-Do-Check-Act cycle is critical atthis point. With a well-thought-out plan in place, you should be ready to launch 
into a successful self-study. 

NDSCS used the following primary steps to complete a successful self-study. Remember though, the process should 
be customized to meet the needs of the specific institution being reviewed. The Self-Study Coordinator kept the team 
on task and organized the details that follow: 

o The NCA Steering Team members who were identified represented a cross-section of faculty, administrative 
positions, and years of service. The Steering Team was given a job description with timelines as well as training 
to familiarize members with the self-study requirements. 

o The data collection process was finalized, and existing committees were assigned specific/appropriate GIRs 
and/or parts of the five criteria to respond to. Steering Team members were assigned as liaisons to each 
committee. Each committee received a common reporting format and a deadline for data submission. 
Reflective questions on each criterion were developed to help committees understand the intent of the 
criterion. 

o The Self-Study Coordinator monitored and supported the committees by providing a common resource area 
and a clearinghouse for questions. Data were reviewed as they were submitted by committees, and it was 
determined whether the information was complete and evaluative in nature. 
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o The Self-Study Coordinator compiled the data into a draft Self-Study Report, which was reviewed by the 
Steering Team and placed on the campus web page for general review and feedback. 

o The self-study was finalized, approved by College Council, and submitted. 

o The importance of communication cannot be overemphasized. Clear expectations need to be established, as 
well as specific deadlines. The campus needs to be kept continuously informed of progress, and their input 
valued. 



The process NDSCS used resulted in a successful accreditation and team visit. It involved a majority of people on 
campus using existing structures and reinforced the importance of data collection for continuous improvement. 
Planning is the key to success within the framework of the institution’s culture and includes the support of 
administration. 



Gloria Dohmart is Director of Assessment and Institutional Research at North Dakota State College of Science in 
Wahpeton. 

Harvey Link is Dean of the Arts, Science, and Business Division at North Dakota State College of Science in Wahpeton. 
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Is It Almost Over or Just Beginning? 
Preparing for the Visit 

Kaylen Betzig 
Ann Marie Krause 



Introduction 

Blackhawk Technical College [BTC) is one of sixteen technical colleges in the Wisconsin Technical College System 
(WTCS). Founded in the early 1900s and accredited by NCA in 1978, BTC continues to serve approximately 18,000 
[5000 credit and 13,000 non-credit) students from south central Wisconsin by providing up-to-date courses and 
programs in technical, basic skills, apprenticeship, and continuing education. While the majority of classes are held 
at Central Campus, classes may also be offered at the Aviation, Monroe, North Rock County, and Beloit Centers. 

In March 2000 BTC was granted continued accreditation with the next comprehensive evaluation in ten years 
following a very successful team visit. The three-year process leading to the reaccreditation was embraced by the 
entire college community and led to renewed energy for continuous improvement. The selection of a faculty member 
and an administrator to co-coordinate the entire process was key to the establishment of a team atmosphere that 
resulted in buy-in and support from all levels of the college. 

The NCA Planning Team conducted initial planning and organization of the self-study process. The team was 
comprised of the President, two Vice-Presidents, two Self-Study Coordinators, a former Self-Study Coordinator, and 
an Institutional Advancement data specialist. This team functioned throughout the process to provide oversight and 
direction for self-study teams as they carried out their tasks. These individuals were key to removing roadblocks, 
resolving issues, and determining recognition. The NCA Steering Team was then selected by the Planning Team to 
provide leadership and direction for each of the twelve self-study teams. Members were selected by length of service 
to the college, diversity, and representation of different levels and divisions of the college. The self-study teams were 
then organized to examine and evaluate the effectiveness of all BTC programs and services. All full-time faculty, staff, 
and administration participated on a study team. Each team presented an oral report to the Steering Team and then 
wrote a report summarizing its findings and providing patterns of evidence for strengths and opportunities for 
improvement. These reports were the basis for the self-study document. 

Based on our experiences as coordinators of the self-study process, the time spent organizing the teams, creating 
and adhering to timelines, and communicating with all levels of the organization and community members was crucial 
to the success of BTC’s reaccreditation visit. We began with the “end in mind.” 



20-20 Hindsight: Reflections on the BTC Road to Reaccreditation 

In the paragraphs that follow we share some of our observations and the lessons we learned in building and leading 
our successful self-study team. 



O Being organized. While it is admirable to involve the entire college, it is unrealistic to expect 100 percent 
participation throughout the entire process. In the times designated as NCA work times there was good 
participation. Overall during the process we had about 65 percent participation. Staff attrition, especially in 
leadership positions, was a concern. Also as new staff members were hired, we had to decide how to involve 
them in the process. Selection of team leaders and members for the different self-study teams was important 
so that a true evaluation of the area(s) was done. Several staff members who had not been previously known 
as leaders were cultivated as team leaders and blossomed in their new assignments. We attempted to involve 
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students on teams, but we did not receive good participation. Although students were invited to be team 
members, often interest fell off qujckly. 

As NCA coordinators we soon learned how important it was to have the support of upper levels of the 
administration. Our president and vice-presidents were very available, listened to our concerns and 
recommendations, and assisted us in any way they could. We were truly empowered to make this process work 
and, therefore, we grew as leaders. It was a wonderful professional development opportunity for us. 

0 Adhering to timelines. Timelines are critical. We made them. We stuck to them and made all teams 
accountable for them. We were successful in maintaining the timelines we originally established. It is 
important to coordinate the number of surveys as well how and when they are administered. It is very easy 
to overload the college staff with surveys requesting input. 

0 Enhancing communication. Communication needs to be centralized. As Self-Study Coordinators, we made 
communication pur responsibility and our priority. We had a plan for communication and understood there 
can never be too much. We use different media to make it unique and fun. Some of the avenues we used were 
radio talk shows; newspaper releases; inservice presentations, games, and activities; email, all-school memos 
on special NCA paper; catchy slogans that appeared everywhere; internal newsletter articles; intranet; 
Internet; presentations to service groups in the community, advisory committees, BTC Foundation, and BTC 
Board; videos; and an Executive Summary of the Self-Study Report. 

0 Maintaining focus. There is a delicate balance between maintaining focus and burning out the staff. Again, 
consistent communication js critical. The college community can get so focused on achieving a ten-year 
reaccreditation that it can lose sight of the college’s primary commitment to continual evaluation and 
improvement. We qsed fun activities, humor, and special recognition to keep the teams moving forward. It was 
also important for us to effectively communicate that this self-study was a college commitment and not the 
Self-Study Coordinators’ project or perspective. College ownership was imperative. 

0 Handling conflicts. Most issues were resolved during Steering and Planning Team meetings. Planning Team 
members served as liaisons to each self-study team to help remove roadblocks, answer questions, and bring 
issues back to Steering Team meetings to be discussed. As Self-Study Coordinators, we were constantly 
mediating. Issues centered on territorial themes, release time, compensation, and clarification of team 
charges. We addressed the conflicts and misunderstandings right away and centered on the process and not 
making it personal. We remained upfront with expectations and looked for win-wins. 

0 Involving the Board. We met with our District Board monthly for a period of fifteen months prior to the visit. 
We reviewed the self-study team reports, emphasized areas on which they should focus, and role-played 
possible interview questions. The time spent with the Board members was well worth it. They became our 
greatest cheerleaders; they learned a lot about our organization; they felt they were an important part of the 
process; and they set the tone for the entire on-site visit. The on-site team made many comments during their 
visit about our knowledgeable board and its passion for what we do at BTC. 

0 Writing the Report. As coordinators, we scheduled time to write in a location that was relatively interruption- 
free. We aljowed six months to write the report, knowing we were also continuing to perform our regular 
assignments at BTC. We searched out examples of good Self-Study Reports, determined the organization of 
the document, and used the self-study team reports as the basis for our content. Getting good, evaluative 
reports from the teams was crucial to our success. Sending out multiple drafts to all interested staff to critique 
to ensure we were keeping the original intent for each self-study team was a priority for us. Still, as we look 
back, there are a couple of areas we would merge. For example, we would combine the chapters on student 
services and academic support as both refer to necessary support services for our students. 

Establishing a single voice for the document actually was pretty easy. As coordinators/writers our writing styles were 
very similar. We enlisted a cadre of editors to help us, and they proved invaluable. We also established early a 
relationship with the person who would be desktop publishing our document (a current staff member who wanted 
this challenge] and subsequently with the printer. This saved us time, money, and many gray hairs. We learned a lot 
about the writing process in general. In retrospect, our document was too lengthy (about 250 pages]; yet it served 
our purposes and remains a workable document as we address the stated opportunities for improvement. The NCA 
on-site team commended us on the evaluative nature of the report. 



ERIC 



Adding an addendum. There is always a time lag when writjng the report. Just before we sent the final 
document to the printer, we added an addendum briefly explaining changes that had occurred. We organized 
the addendum by chapter and bulleted the main items. The NCA on-site team found this section helpful, 
because it was evidence that BTC was already moving ahead with making changes based on the self-study 
team findings. 
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0 Using technology. We discovered that using technology expedited communication, though it shouldn’t be 
the sole method used. We placed drafts of the self-study on the intranet for review, conducted surveys on the 
Internet, and placed third party comment notices on our web page. In addition, we had a 20 minute video 
produced explaining the mission of our college, introducing our administrative team and divisional areas, and 
highlighting what weather, amenities, etc., the on-site team should expect as they arrived at our campus. The 
NCA on-site team found this video to be especially effective in preparing for their visit. We also used video 
technology to explain the NCA process to our staff and advisory committees. While time consuming, these 
activities were well worth it. 

During the actual visit, technology was made available in the resource room [fax, phones, computers, printers, 
and a TV/VCR). Also, we assigned a technology staff person to be available to assist team members as needed. 

0 Finding those Self-Study Coordinators during the visit. Looking back, we jokingly say we were on call 
24/7. While that is not necessarily true, we can say we were pretty much on call from the time the team arrived 
on campus Sunday evening until they left at noon on Wednesday. We had cell phones, and the team members 
called us frequently with questions; requests for information; requests to set up interviews, meal reservations, 
etc. Accessibility was the key. and again the on-site team commended ns on our approach and promptness 
to their requests. We established a work area for ourselves near the resource room and were the first point 
of contact for all of their requests. We arranged to have someone in every department and division on alert 
to help us find specific materials. Our goal was rapid response. 

0 Building a resource room. Although the resource room is officially compiled right before the visit, one must 
keep its organization in the forefront of one’s mind at all times. We started building the resource room right 
after we finished the writing process, while the evidence we needed was still fresh in our minds. The best thing 
we did was to require the self-study teams to submit patterns of evidence (POE) with their written reports. We 
were then able to pull the things we needed from theirfiles and make copies. We organized our bins by chapter 
and cross-referenced duplicated POEs. We also bookmarked patterns such as curriculum and web pages on 
computers that we had in the resource room. Our resource room was centrally located on the main campus. 
We were able to split a meeting room in half, reserving one half as a work area and the other as a break area 
that could also be used for one-on-one interviews. We also had a work room/meeting room available at the 
hotel that was set up with computers and printer, supplies, and refreshments. 

0 Building an on-site agenda. The on-site team chair establishes the agenda for the team visit. We used email 
to work with our team leader to adjust interview schedules, the Board dinner, outreach center visits, etc. The 
process was effective, and the team adhered to the schedule. We provided goodies in the hotel rooms for all 
team members as they arrived. The Self-Study Coordinators and the college president greeted the team 
members at the hotel the first evening before the dinner with the District Board. This put a personal touch on 
their arrival. Every team is different, so we established contact with our team leader through our college 
president soon after we knew the composition of the team. 

0 Sending materials to the team. We followed NCA's guidelines for sending materials to the visiting team. 
In addition to having the required items, we provided some extras that the team appreciated. In the packets 
we included the “Get to Know BTC” video, maps of the area, and hotel brochures. Team members knew exactly 
what to expect on arrival. 

0 Taking time to say “Thanks.” As Self-Study Coordinators we believed recognition, no matter how simple, 
was important We decided not to wait until the end to say thank you. Our college supported this recognition 
of efforts throughout the self-study process. Every college employee received NCA 2000 note cubes, BTC 
mouse pads, new nametags, and team T-shirts. We surprised self-study teams with treats during team 
meetings. Team leaders also received etched appreciation plaques. Since all self-study team members had 
done their jobs well, the exit interview with the on-site team resulted in smiles all around and a time of rejoicing. 
Therefore, following the visit, the BTC Foundation sponsored a celebration pasta lunch for the entire staff. The 
Self-Study Coordinators served the lunch. 



Where the Road Is Leading Us: Is It Almost Over or Just Beginning? 

Having successfully completed a self-study and gained a ten-year reaccreditation, BTC congratulated itself and then 
went to work on the recommendations, preparing for the next visit in 2009-2010. The self-study findings have been 
incorporated into strategic and work plans for all areas of the college. We are using the self-study as a benchmark 
and have created a system to monitor changes that are being made. Thus, the college is positioning itself for continual 
improvement and evaluation. The college has made a commitmentto keep the energy alive for improvement. The Self- 
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Study Coordinators, though tired and worn, are committed to monitoring progress, and if asked, would say, “Yes, we'll 
do this again!” 



Kayien M. Betzig is the Manager of the Institutional Advancement Department at Biackhawk Technical College in 
Janesville, Wisconsin. 

Ann Marie Krause is Dean of the General Education Division at Biackhawk Techncial College in Janesville, Wisconsin. 
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Designing an Integrated 
Non-lntrusive Self-Study 
for Your Institution 



John Speary 

When Butler County Community College began its self-study process in the spring of 1997, one of the individuals 
selected to serve as Self-Study Coordinator had worked as a subcommittee co-chair in the self-study conducted ten 
years earlier. Reflections on the difficulties of that earlier process and insights gained from self-study mentoring 
received at the NCA Annual Meetings yielded some helpful principles that made the successful self-study completed 
in 2000 a much more positive and less painful experience for all involved. The primary purpose of this self-study 
mentoring session will be to share those reflections and insights with other Self-Study Coordinators who are still early 
on in the process. 



The Challenge 

Picture this: it has been seven (possibly eight) years since the last NCA Team Visit to your community college, which 
resulted in a successful ten-year extension of your institution’s accreditation. In three (possibly two) short years, the 
next Team Visit will be on the institutional horizon. The time has come to begin a formal institution-wide self-study 
process again, which will yield the required self-study document and eventually another much desired ten-year 
extension to your accreditation. The Commission’s requirements are stated clearly and succinctly. Your college must 
examine its effectiveness, assess the strengths and concerns found in all segments of its operation, and plan howto 
make the most of those strengths and address those concerns. The self-study must focus on the full range of the 
institution’s mission and purposes. Because the self-study should look at the entire institution, it is crucial that the 
process involve as broad a range of the college’s constituents as possible. You have been chosen as co-ordinator for 
the self-study process. 



A community college, armed with relatively limited resources of people, money, and time, and faced with the 
formidable task of conducting a self-study, may find the challenge almost overwhelmingly intrusive. The normal 
operations of the college become congested with an elaborate network of temporary activities that seem to exist only 
for the sake of the self-study process. The complex efforts to create required patterns of evidence that will document 
the college’s achievement of the five criteria may seem disconnected from the regular ongoing operations of the 
institution. If the college leadership has had the foresight to create and maintain an ongoing process of perpetual self- 
study (that meaningfully references the NCA criteria) through its institutional effectiveness assessment and/or 
program review, then the task of the self-study required by North Central will not be so daunting. However, such 
foresight is not often the case. If the self-study process is rather seen as a once-in-a-decade task to satisfy 
accreditation requirements, the monolithic challenge of accomplishing a successful self-study can become painfully 
intrusive to the regular functions of the community college. 



Seemingly, the burden of this task, at least initially, lies at the feet of the individuals) who has been designated as 
Self-Study Coordinator®. That individual may have never experienced such intensive complexities as involved in 
facilitating such a far-ranging and, in many senses, ambiguous project. The Self-Study Coordinator may be 
overwhelmed by a sense of personal responsibility in trying to corral such a mysterious phantom beast, the 
institutional self-study. The coordinator can save herself a great deal of grief while helping the institution avoid a 
number of common, perfectly understandable, but unfortunate pitfalls along the path toward the completion of an 
effective, useful, and thereby successful self-study. 
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Creating an Effective Steering Committee 

Steering committees that seem to represent only certain parts of the institution, or that are overloaded with 
administrative personnel, or that are more of a group of dignitaries than working individuals will not be effective in 
moving the self-study process along. The Self-Study Coordinator should work with the school’s administration to 
recruit a large, genuinely broad-based steering committee made up of respected hard working leaders from all areas 
of the college’s constituency. The steering committee roster should contain representatives from all levels of the 
institution’s operation: 

o Various strata of administrators 
o Faculty from all academic and vocational divisions 
o Staff from all ranks of student services and instructional support 
o Operational employees who maintain the day-to-day mechanics of the college 
o Members representing different segments of the student population 
o Individuals from the larger community impacted by the college’s operation. 

Careful consideration of the mix of personality types and working styles must impact the selection of steering 
committee members. There is an enormous diversity in the tasks involved in the completion of a self-study. The 
steering committee membership should consist of individuals with an equally diverse range of talents. A proven 
willingness to contribute in active and meaningful ways to given tasks must be the most basic trademark of anyone 
considered an appropriate selection for inclusion on a steering committee. 

The size of the committee can grow too large to be manageable in scheduling meetings. However, merely being 
smaller and more easily scheduled is not necessarily better. The task of the self-study is very large and broad in scope, 
and even at the leadership level many well-chosen hands make much lighter the work. A shrewd appreciation of the 
mania of activity that is characteristic of the contemporary educational community can help in creating an effective 
steering committee. Scheduling a meeting that everyone can attend is an almost impossible goal. A productive 
steering committee can operate on a “buddy system” from its very inception. If a pair of committee members heads 
up each subcommittee or task area from the start, the chances are much greater that at least one representative from 
each task area will be able to attend any given meeting and keep that area actively in touch with the larger body on 
a regular basis. 

The coordinator must focus a good deal of attention early on providing necessary and sufficient orientation and 
training for steering committee members so that they are familiar and at least reasonably comfortable with the 
demands and benefits of the tasks they face. A thorough familiarity with the NCA Handbook and representative self- 
study documents from other schools must be a priority for the coordinator entering into the process. The steering 
committee as a whole should familiarize themselves with the NCA requirements for accreditation and guidelines for 
self-study. Easy access to the Handbook of Accreditation is a very good thing. Steering committee members who are 
exposed to self-study documents from comparable institutions will have a clearer understanding of what they are 
about and will have more confidence in pursuing the goal of the useful self-study. An effective Self-Study Coordinator 
will work with the steering committee to construct specific job descriptions that clarify and facilitate the effective 
delegation of reasonable shared responsibility throughout the self-study process. Unquestionably, most persons feel 
much more comfortable when they can clearly see the target they are trying to hit. 



Pacing the Process 

The elusive character of time is not different in the world of higher education than in any other. Two or three years 
rush by before you can believe it possible. A Self-Study Coordinator must insist on the creation of a feasible long- 
range timeline for the self-study process. That timeline must realistically reflect the demands of the project alongside 
the daily dynamics of the institution’s operations. Those who generate such a timeline must recognize how the 
institutional culture functions in terms of the completion of college-wide tasks. An unrealistic timeline that aspires 
to the ideal and ignores established patterns of behavior among the self-study participants is doomed to futility. 
However, planners must allow for flexibility within the projected deadlines for different stages of the self-study 
process. Crises will inevitably arise that force unavoidable delays so the timeline must not be designed to plunge the 
participants into a last-minute panic when such delays occur. A reasonable timeline that reflects the flow of the 
college and paces the process over an extended period of time will help integrate the self-study effort into the day- 
to-day operation of the institution in a nonintrusive way. 
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If those constructing the timeline shape it as an action plan, including not only the tasks to be completed and their 
projected dates for completion but also the responsible parties for each task, they will find the timeline to be a more 
meaningful and effective tool. The steering committee should see a close relationship between the job descriptions and 
the timeline its members produce. Successful coordinators will review progress with the steering committee periodically 
so they can refine and republish the timeline, putting particular emphasis on celebrating the completion of different 
stages of the process. Regular updates to the college community at large keeps the goal in focus and the efforts of all 
involved on task. A coordinator who can exert firm encouragement while expressing compassionate flexibility regarding 
the inevitability of reasonable delays will receive desired results in a much more relaxed working atmosphere. 

Structuring the Self-Study 

The self-study document must present patterns of evidence that demonstrate the manner in which the college 
satisfies the five NCA Criteria for Accreditation. The steering committee must decide how best to approach that task. 
The strategy and structure of the self-study process will ultimately determine the organization and focus of the self- 
study document and should not evidence only the fulfillment of the five criteria. Both the process and the document 
should clearly fefieci; Liie particular character of the college being studied. A coordinator should lead the steering 
committee through a careful analysis of the existing organizational structure of the institution. This analysis will find 
the correlation between the accreditation criteria and the mission and purposes of the various departments, 
programs, and service units of the college. For instance, the activities of certain departments in the college such as 
Human Resources, Accounts Payable, and Buildings and Grounds most directly address the issues raised by Criterion 
Two. Existing academic assessment programs and institutional effectiveness offices pertain to Criterion Three, and 
so forth. A self-study will make more sense to the realities of the college if it is structured first around the existing 
architecture of the organization and then correlated to the criteria presented in the Handbook. 

Steering committee members will function most efficiently if they are organized as co-chairs of subcommittees 
sharing the various responsibilities for research and documentation. Each subcommittee will focus on one functional 
dimension of the college’s operation. The subcommittee will either look at existing documentation or generate new 
documents to produce the patterns of evidence necessary to address the relevant criteria for its focus area. As the 
process progresses, subcommittees will produce reports that reflect a careful critique of existing operations within 
that area of the college. These subcommittee reports then dovetail together, easily forming the major body of the final 
Self-Study Report. The Self-Study Coordinator or whoever serves as the writer/editor of the document will be able 
to construct the final document from the rich resources the steering committee provides. There does not need to be 
any sense in which the writer/editor of the final document is starting from scratch. 



Utilizing Human Resources 

A wise college administration will endorse the broadest possible base of participation in the self-study by the 
employees of the institution. All full-time employees should have the opportunity to make some contribution to the 
process. Subcommittees made up of personnel employed in the area being studied working alongside representatives 
from all other sectors of the college will be able to analyze the operation of each area in an informed, positive, yet 
critical manner. The likelihood that there will be total universal participation is not great. However, the more people 
across the institution who are involved, the more the self-study will seem to be part of the fabric of the institution; 
ultimately, the more useful the observations made in the self-study will be. If a major portion of the college 
constituency is involved, there will be greater sense of ownership in the project. The whole college community should 
be exposed regularly to the progress of analysis and given opportunity for input. 

The Self-Study Coordinator must realistically assess the strengths and weaknesses of the workers involved in the process. 
One of the workers whose strengths and weaknesses must be assessed most accurately is the coordinator. Some leaders 
are best at organizational planning, some at facilitating meetings, some at structuring research efforts, and so forth. Most 
individuals chosen as coordinators will have many varied strengths, but will not have all strengths. The effective 
coordinator will capitalize on his or her own strengths and look to others more equipped to bolster points of weakness. 
For example, the coordinator may or may not feel well-equipped to handle the writing/editing of the final document. Most 
coordinators will find it best to share this responsibility with at least one other person with appropriate expertise. 

Addressing the Demands of Research 

Another area of weakness for the coordinator may be in handling the kind of systematic research that accreditation 
self-study demands. If this is the case, the coordinator will certainly not be alone in this concern. Research of the 
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caliber called for in a self-study is often an intimidating task for many who will need to be involved in the self-study 
process. The broad range of programs and services that must be assessed is daunting. The volume of research 
documentation that results from such assessment is formidable. When a community college employee looks at 
another institution’s representative self-study document that document has an appearance comparable to a major 
thesis or dissertation. The seeming magnitude of the task as compared to the normal daily activities of that employee 
may prompt the self-study participant to feel unequipped for the research task. 

If the coordinator is not a research specialist, he or she should engage in-house experts on research methods ( a 
director of institutional research, if available] to help design plans for conducting the necessary research. Members 
of the college constituency who would not ordinarily consider themselves researchers will be much more comfortable 
doing such work when carefully guided by individuals with much more research experience. Carefully constructed 
research plans that lead the participant through a systematic process of document analysis serve as an excellent 
supplement to the comforting guidance received from the initial job description. The whole function of steering 
subcommittees benefits from the formulation of overall research plans that direct their efforts. A research specialist 
serves as an invaluable resource for the structuring of patterns of evidence addressing the accreditation criteria. Such 
a research expert can also help the leadership of subcommittees recognize existing documentation that will serve 
their purposes. Self-study participants should always avoid generating new documentation when records that meet 
their needs already exist. 



A Crucial Leadership Strategy 

An effective steering committee coordinator must never neglect one particular leadership strategy: delegate, 
delegate, delegate. Self-study coordination is not a healthy place to employ micromanagement. You can not take care 
of everything. You should nottrytotake care of everything. You can, however, seek to bring together a talented diverse 
company of workers who can agree on a reasonable time frame and strategies to accomplish their varied tasks. At 
the beginning of the self-study, you should work to help your staffers feel as equipped and aware of their specific 
tasks as possible. As the process progresses, you can communicate with them to oversee, encourage, and, hopefully 
at most, troubleshoot to solve problems when they unexpectedly but inevitably arise. If the Self-Study Coordinator 
does not entrust the majority of the task to other talented, dependable individuals, the self-study will be an utterly 
intrusive element in the life of the coordinator, if not the college as a whole. 



A Final Observation 

It would seem that the fewer people actively involved in the self-study process, the less intrusive the process would 
be to the ongoing operation of the college. However, a self-study that grows out of limited participation will not project 
an accurate picture of the character of the institution. The efforts of a small group of workers who gather information 
for analysis on a “hit and run” basis will seem all the more intrusive to the larger college constituency who have no 
clear point of contact with the self-study process. The conclusions of such a self-study would enjoy no ownership on 
the part of the larger college community and would, therefore, be ultimately of very little use to the future of the 
institution. A Self-Study Coordinator will best serve the college and the self-study process by encouraging as broad- 
based and well-informed participation in the task as possible. 



John Speary is Director of Academic Assessment at Butler County Community College in El Dorado, Kansas. 
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Using Your Mission Statement 



Vickie Hess 



Introduction 

An institutional self-study is a massive undertaking. The criteria are general and flexible— deliberately so, in orderthat 
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clientele, and purposes. However, to those charged with the responsibility for producing a self-study, the generality 
and flexibility can be daunting. While suggested patterns of evidence provide some clues, they do not provide the 
organizing theme for what will in all likelihpod be a 200-300 page document. The process of self-study is not merely 
a checklist of standards [although those minimal standards do exist, as reflected in the General Institutional 
Requirements). It is, rather, an opportunity for an institution to ask itself the basic identity questions: Who are we? 
What are we trying to accomplish? Do we have what we need? Are we accomplishing our purpose?, and can we 
continue to do so? Are we being true to our own sense of institutional identity? The “who are we” question is answered 
by the mission statement and made concrete by the statement of purposes derived from that mission. The mission 
and purpose statements provide structure, helping to focus attention on the rnajor issues to be addressed within the 
document and making the document cohesive despite the diversity of activities examined in the self-study. 



Source of the Mission Statement 

There are two ways to view an institutional mission statement. Many of those who work at our institutions regard the 
missjon statement as nothing more than a statement of ideals suitable for a catalog. The NCA, however, believes that 
we mean what we say in the mission statement, and bases the whole self-study process and team visit on what we 
say about ourselves. From this perspective, the mission statement and the purposes derived from that statement can 
provide dynamic guidance for institutional growth and development. The self-study process is an opportunity for an 
institution to reexamine its mission statement, change it if necessary, and use it as a gpide for institutional 
improvement. Organizing the self-study plan from the missjon and purposes ensures that the criteria are addressed, 
and that the evidence collected in the self-study process is relevant to the needs of the institution 

The mission statement is a relatively short general statement of the institution’s reason for being. Typically, the mission 
has significant continuity with the history of the institution. If the mission statement is to serve its purpose, it must 
reflect the institution’s current ideals and identity. It might be necessary to lead the institution through the process 
of determining whether it does so, and revising the statement if need be. In our previous self-study, the revision of 
the missjon statement was the first task undertaken. The changes made then were not substantial. However, the 
process of discussion, at least among the faculty, reaffirmed commitment to a well-defined mission at a time of 
significant institutional stress. 

The most recent self-study came after a period of dramatic growth and on the heels of a two-year strategic planning 
process during which the mission statement was revised. While faculty and other constituents were represented on 
the task force and were consulted via polls and focus groups, it was the Strategic Planning Task Force that proposed 
the wording of the mission statement. After the proposed statement was adopted by the Trustees, the administration 
and task force initiated an all-out effort to develop a sense of ownership among institutional constituents. The 
development of the mission statement and identification of priority areas was followed by the appointment of working 
groups to explore the implications of the new statement for academics, student life, financial affairs, and assessment. 

Regardless of how the mission statement arises, and regardless of whether it is revised at fhe time of the self-study, 
it is important that the majorconstituencieson campus focus attention on the institutional mission. Before a self-study 
is complete, every office on campus has had input and has been asked for information; understanding of and 
ownership of the mission statement helps keep the internal research focused even before the document authors 
begin their work. 
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Institutipnal Purposes 

If the requirement of institutional purposes did not exist, there would still be a need to develop a set of concrete 
statements of desired outcomes. At our institution, these were not developed with the mission statement; the vision 
and core values statements were too general to be used as purposes. They were instead developed by the Assessment 
working group in the implementation phase of the strategic planning process, and they were developed specifically 
because the Self-Study Coordinator insisted that they were needed for the NCA process. However, the purposes On 
parallel with other internal documents] did help us solidify the new mission statement by forcing us to answer the 
questions: What will this institution look like if we fulfill this mission? What will our graduates look like? The purposes 
yvere broad enough so that every facet of campus life was addressed, but specific enough so that each area could 
recognize and address relevant purposes. 



Creating Community 

The maintenance of community at our institution has been a challenge for two reasons: quantitative growth and 
program diversity. In the decade between the previous self-study and the one just completed, the traditional student 
body nearly doubled and the faculty increased by nearly 50 percent. Nontraditional programs grew at even greater 
rates. The accompanying increase in numbers of employees with faculty status has, over the years, forced us to change 
the traditions that made the smaller institution a cohesive “family." Faculty dinners gave way to dessert receptions; 
meetings of the full faculty became sparsely attended; finding affordable locations for a fall retreat of the entire faculty 
became unmanageable. Clearly, community was no longer going to come about without intentionality. 

Our institution grants degrees through two very different types of programs: traditional, campus-based programs 
serving mostly (but by no means entirely] 18-22-year-old resident students, and nontraditional programs in a variety 
of formats throughout our home state that address the needs of working adults seeking to extend their education. 
From its inception, the institution has recognized the need to find ways to maintain unity and community while 
allowing the Adult and Professional Studies Division the freedom to innovate to meet the needs of its constituency 
in a rapidly changing educational climate. While unity and diversity result in creative tension, focus on our mission 
and purposes has provided a common bond. While the Adult and Professional Studies Division programs have a 
different “look and feel” from those of the traditional campus, they seek to address and assess the same institutional 
purposes. 

Community can no longer be based on our all knowing each other, our all working in the same geographic location, 
or even (except in the most general sense] our all doing the same thing. However, a strong sense of common mission 
does provide the glue to bond us into one institution. In the context of self-study, it was by using the mission and 
purposes as the major theme that we were able to demonstrate that we are truly one university. 



Information Gathering 

The information-gathering process can pose a number of challenges: 

1. Some administrators are “big picture” people, and have a hard time getting past general statements and 
providing the necessary detail and documentation. Even with a representative steering committee, it is 
possible that some programs will be neglected entirely because no one knows to ask about them, and the 
appropriate administrator never mentions them. 

2. Other administrators will provide overwhelming amounts of detail with no clear focus. 

3. There is always a temptation to provide overly optimistic assessments of the performance of one’s own area, 
without sufficient documentation. 

4. There is also a temptation to use the self-study process to provide leverage with administration by claiming 
as “needs” or “challenges” the entire wish list of an area. 

While none of these challenges is completely unavoidable, they can be reduced by providing clear structure and 
getting responses from those with direct responsibility for the program. In our case, we asked for responses from the 
directors of administrative areas and from the chairpersons of academic divisions (or program directors in Adult and 
Professional Studies). 
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The structure that we provided asked each unit to begin with a set of three-five goals. They were to explain how their 
goals fit the institutional purposes. From there they could tell us what major changes had occurred since the last self- 
study; what their strengths, challenges, and opportunities were; and how they assessed these. Where appropriate, 
particularly in academic areas, we requested assessment and other data. We emphasized the need to document any 
claims that were made. Having the units begin by tying their goals to the institutional purposes gave these preliminary 
reports a focus that they might not otherwise have had. The focus on purposes also provided some boundaries to the 
collection and presentation of data; it was not necessary to collect every possible piece of information, but rather to 
collect information needed to support the response to the purposes. 



Writing the Document 

One of the major difficulties in actually writing a self-study is making sense of the vast amount and diversity of 
information. The criteria can provide a basis for organizing the document, but some of its sections will be large and 
involved. For us, the mission and purposes provided the framework for pulling togetherthe information into a coherent 
document. 



Conclusion 

From the perspective of the accreditation process, the institutional mission and purposes form the identity of that 
institution. NCA is not prescriptive in defining what a “good” college or university is; rather, we are asked to define, 
within the very broad traditions of higher education, what type of institution we mean to be and what particular kind 
of contribution we mean to make to the common good. The mission and purposes are our self-definition. By using 
them to drive the self-study process, we can enhance our communities and make the process and the product 
meaningful to the institutions we serve. 



Vickie Hess is Chair of the Division of Natural Science and Math at Indiana Wesleyan University in Marion. 
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Structuring and Conducting 
a Self-Study: 

Achieving Comprehensive 
Institutional Evaluation 



Anne C. Dema 
Nina T. Pollard 



Overview 

The purpose of the session is to facilitate a discussion about how an institution can structure and conduct a self-study 
process that allows for a comprehensive evaluation resulting in improvements in institutional quality and perfor- 
mance. William Jewell College completed a two-year study [begun in June 1 998) that culminated with an on-site visit 
by an evaluation team in November 2000. The team recommended that accreditation be continued, and that the next 
comprehensive visit should occur in 2010-1 1 . 

Like most institutions involved in this process, William Jewell was challenged to conduct a self-study that ensured 
o college-wide participation, 

o realistic evaluation of every aspect of the institution, 

o identification of patterns of evidence that demonstrate the institution’s compliance with the Criteria for 
Accreditation, 

o generation of clear and concise report findings, and 

o accurate dissemination of information to the various campus constituencies. 

Jewell’s desired outcomes were to attain continued accreditation and to improve its overall institutional quality and 
performance. 

At the beginning of the self-study process, the above challenges coupled with the desired outcomes seemed 
overwhelming. As an institution, we wanted to conduct an honest and thorough self-study, yet we were aware of the 
natural tension associated with using this process to improve the college and at the same time trying to make our 
case to the North Central Association (NCA] for continued accreditation. The college successfully implemented a self- 
study structure and evaluation method that allowed us to accomplish the above challenges and to attain the desired 
outcomes. This paper provides an overview of the self-study process used by the college and the evaluation method 
implemented by the steering committee to examine the institution in light of the five criteria established by NCA. In 
the self-study mentoring session, participants will apply the evaluation method to information gathered during the 
self-study process. It is anticipated that this activity will spark conversation about the benefits and challenges 
associated with this evaluation method and self-study process. 
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Profile of the College 

William Jewell College is a private, four-year liberal arts college located in Liberty, Missouri, adjacent to the 
metropolitan comm unity of Kansas City. The college is primarily residential, with a campus of approximately 500 acres. 
Founded on the western frontier in 1849, William Jewell College’s mission is to 

o provide students with a liberal arts education of superior quality; 

o serve communities beyond the campus educationally, culturally, and socially; and 

o be an institution loyal to the ideals of Christ, demonstrating a Christian philosophy for the whole of life, and 
expressing the Missouri Baptist heritage which is the foundation of the college. 

The college is committed to providing an education that liberates and enhances a person’s intellectual and spiritual 
life while at the same time is connected to practical application in work and service. Since 1994, the Carnegie 
Foundation forthe Advancement of Teaching has classified William Jewell as a “Baccalaureate I” institution. In August 
2000, it was reclassified as a "Baccalaureate Colleges-Liberal Arts” institution. 



Self-Study Structure 

The self-study process began in the summer of 1998 when the President and Provost named a faculty member as the 
institutional Self-Study Coordinator. The Provost and the Self-Study Coordinator identified a team of faculty and staff 
representing not only persons with the requisite skills and wisdom, but also the various college subcultures to serve on 
the steering committee and self-study teams. The steering committee was composed of five faculty members (four 
tenured and one untenured) and two college staff members. Ex-officio members included the President, Provost and 
Vice President for Academic Affairs, the Chair of the Board of Trustees, and the Chair of the Alumni Board of Governors. 

In an effort to address NCA’s five criteria, to give attention to critical areas of self-scrutiny at Jewell, and to build on 
the work of strategic planning and other recent initiatives, the self-study efforts were coordinated around four areas. 
These areas included 

o academic programs, 
o institutional integrity and values, 
o resources management, and 
o student development. 

The President, upon recommendation of the steering committee, appointed members of the college community to 
self-study teams focusing on these four areas. Each team consisted of a chairperson, five or six faculty and staff 
members, one or two members of the steering committee, and pertinent ex-officio members. In addition, a self-study 
team, the Institutional Data Clearinghouse (IDC), was created to oversee all information needs during the process. 
The IDC team was responsible for overseeing all research and data management related to the self-study. The impetus 
for creating this team was to promote efficiency and cooperation and to help avoid redundancy and overtaxing of 
institutional resources. 

The four self-study teams were charged with examining the respective areas of importance and providing evidence 
that the institution was meeting the General Institutional Requirements and the Criteria for Accreditation as outlined 
in The Handbook of Accreditation. Each self-study team determined the best way to identify information needs, 
evaluate the collected information, and structure the results of the study into a team report. The self-study team 
reports contained a description of the process used to conduct the self-study, an identification of institutional 
strengths and areas needing attention, and an evaluation of the current reality. The steering committee members 
worked as liaisons with the self-study teams to help coordinate information needs and to provide guidance and 
encouragement to the team members. Each self-study team was asked to submit both an electronic and a hard copy 
of their report (and all supporting materials) by June 1, 1999. Every department, office, program, administrator, and 
trustee member contributed to the self-study process. 

Two task forces were also created. One consisted of veteran faculty members who worked during the fall of 1998 
examining the 1990-1991 Self-Study Report and the 1990-1991 Visiting Team Report. This group was charged with 
identifying key issues from these reports for the steering committee. The task force was asked to submit a report by 
January 1999. 
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During the summer of 1999, the steering committee also appointed a task force composed of students who were 
known leaders on campus. The student task force led other student leaders at Encampment in August 1999 in the 
identification of issues important to students. During the fall, the student task force developed tools for measuring 
student perceptions on these issues, collected data, and analyzed the results of the study. The task force was asked 
to submit a report of its work by February 15, 2000. 



Steering Committee Efforts 

The steering committee met approximately every other week during the 1998-1999 academic year. The focus of its 
work was to provide updates about the self-study team efforts, identify common information needs, discuss any 
submitted team reports, and begin developing an outline for the self-study report. The Provost attended the majority 
of steering committee meetings, and the other ex-officio members were kept informed via distribution of meeting 
minutes. During the summer of 1999, the steering committee met at three all-day retreats to discuss the progress of 
the self-study process and to develop a more detailed strategy for the second year of the self-study. 

During the 1999-2000 academic year, the steering committee met on a weekly basisfortwo-hour meetings. By August 
1999, all self-study team reports had been submitted. The steering committee focused its efforts on discussing and 
reviewing recommendations in each report. To ensure a thorough evaluation of the information contained within 
these reports, the steering committee members used the following questions as a guideline for their discussions: 



Descriptive: Is a complete description of each area provided? 

What resources [catalog, self-study, pamphlets, etc.) describe the program/department? 
Which resources need to be collected for the Resource Center? 



Assessment: What methods of evaluation are used in the program/department? 

What is the relationship between the stated goals and the goals of the institutional strategic plan? 
What evidence does the program have to demonstrate its effectiveness? 

What action(s) is(are) taken when concerns are identified? 

Issue-based: What issues of concern does the report raise? 

What actions have been taken to address the concerns? 

What issues of concern identified by the steering committee are not raised in the report? 
What additional information needs to be collected? 

What actions does the committee think need to be taken? 



Next step: What recommendations should the steering committee make to the administration? 



After evaluating all of the self-study team reports using the above questions, the steering committee worked with the 
college community to collect any new information. This new information was then reevaluated using the same 
process. By early spring 2000, the steering committee had thoroughly discussed every self-study team and taskforce 
report. In April 2000, the steering committee began synthesizing issues of importance from all of these reports and 
reevaluating the organization of the Self-Study Report. 



Self-Study Report 

Initially, the steering committee conceptualized that the structure of the Self-Study Report would reflect the structure 
of the self-study process. We thought the final report would be organized according to issues raised in the areas of 
academics, institutional integrity and vitality, student development, and resources. Naively we thought the team 
reports could be put together to generate the final report (with appropriate editing). However, as a result of our 
evaluation of the individual reports, institutional, not just “area," issues surfaced. We found that issues identified in 
one self-study team report were often raised in another, and in some instances concerns or points raised in more than 
report indicated the presence of a more pervasive institutional issue. This reality coupled with the recognition that 
the final report had two purposes (to attain continued accreditation and to improve overall college quality and 
performance) and two corresponding audiences (members of the evaluation team and the various constituencies of 
the college) caused us to revise the organization of the final report. 
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The Self-Study Report is organized into five parts. The first part (Chapters 1 and 2) provides information about the 
college's profile and accreditation history and the process used to conduct the current Self-Study Report. The second 
part (Chapter 3) provides evidence that the college is fulfilling the General Institutional Requirements as identified 
by the North Central Association of Colleges and Schools. In the third part (Chapters 4-8), the college provides 
evidence that it is in compliance with the five Criteria for Accreditation. In these chapters, the college presents 
information regarding its mission and purpose (Chapter 4), its organization of resources necessary for accomplishing 
these purposes (Chapter 5), its achievement of the stated educational and other purposes (Chapter 6), its 
preparedness to maintain these patterns in the future (Chapter 7), and its ability to interact with internal and external 
constituencies with integrity (Chapter 8). Part four is a summary that articulates Jewell’s strengths and challenges 
as identified during the self-study process (Chapter 9) and the college's request for continued accreditation (Chapter 
1 0). The fifth part contains the Basic Institutional Data forms and other appendix material. Chapter 9, “Learning from 
the Self-Study," presents issues that emerged during the self-study steering committee’s deliberations as a result of 
reading numerous documents, listening to campus constituencies, analyzing and evaluating data, and participating 
in extensive conversation. We hoped that the information in this chapter would spark campus conversation as Jewell 
seeks to progress and develop as an institution. 

in the seif-sludy mentori ny 3633 icn, ws wi!! discuss the relationship between the outcomes of the self-study process 
and the Self-Study Report and the articulation of the college’s strengths and challenges in more detail. We wiii aiso 
share some of the benefits experienced by the college as a result of the comprehensive evaluation method used during 
the self-study process. 



Anne C. Dema is Associate Professor and Chair of Chemistry at William Jewell College in Liberty, Missouri. 
Nina T. Pollard is Provost and Vice President for Academic Affairs at William Jewell College in Liberty, Missouri. 
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Interdependence, Collegiality, 
and the End of Democracy: 
An Institutional Strategy for 
Framing the Self-Study 



Richard H. Hanson 
Sally C. Crisp 



Interdependence 

The self-study process was approached as an opportunity to reinforce the collegial culture of this university and also 
to raise awareness of the university’s mission. The university has a long history of participatory democracy by faculty 
and staff in its affairs. Faculty and staff, many of whom were hired during the formative years of the university, created 
the system of governance and started engaging the university’s communities in ways that advance the university’s 
unique mission. Interdependence among units, a minor issue in the two previous NCA self-studies, was developed 
into the central theme for this self-study. 

Units within the university have developed their own missions and programs that support the more encompassing 
mission of the university. Therein lay the challenge. These units, working their part of the larger whole, may have 
become unaware of their connectivity to and dependence on other units within the university. 

Prior to and during the early stages of the period when subcommittees were performing their assigned duties for the 
self-study, the entire university, faculty and staff, participated in two data-gathering exercises to identify and measure 
the strength of interdependent working relationships among units. By making this the premier universal activity for 
the campus while organizing for the self-study, the message was sent that interdependence among units would be 
examined. The results of this data-gathering activity were summarized into a report that became a primary source 
of data for the subcommittees. More detail about this process is given at the end of this paper. 



Collegiality 

The self-study coordinator and university provost initially agreed how the report was to be organized and which 
developments and issues were going to receive special attention. This top-down decision on how to frame the 
presentation of the report helped the members of the 17 issue-focused committees and the eight college/school 
deans concentrate on their responsibility to write their reports. 

Invitations to participate in the self-study process or to nominate colleagues to do so were sent from the provost to 
all members of the faculty, key members of the staff, and student leaders. The self-study coordinator and provost 
selected subcommittee leaders and identified and placed most of the members who ultimately served on the 
committees. Ninety-three faculty, staff, and students participated in the work of the subcommittees. 



ERTC 



Subcommittee reports were submitted to the self-study coordinator approximately 1 0 weeks after the beginning of 
this phase of the process. This corresponded to the end of the spring semester, giving the self-study coordinator the 
summer to evaluate the reports and prepare them for the next stage in the process. After consulting with 
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knowledgeable campus leaders, it was determined that some of the reports were insufficient. For example, when a 
strong personality was a subcommittee chair who was responsible for writing the report, it appeared that neither data 
nor counsel of their subcommittee members influenced what was written. A couple of subcommittees were unable 
to report anything. 

The 15-member self-study taskforce, a carefully selected group of faculty, staff, and students, received 25 separate 
reports shortly after the start of fall semester. Its work was finished by the end of November. 



The End of Democracy 

Convening the self-study task force, the highest level of collegial review in the process, signaled the end of democracy. 
The many voices in the subcommittee reports had to be melded into a single university voice. 

The task force began having success assembling the final report only after it developed consensus about what to say 
and how to say it. Paper versions of the 25 reports from the subcommittees and deans stacked up nearly 1 0 inches 
high. Expressed very succinctly, the task force agreed it had to “tell the story.” The story was that “the University of 
Arkansas at Little Rock [UALR) is Arkansas’ only metropolitan university, and that UALR is meeting its mission 
objectives, and has the resources and organization to continue to meet its objectives.” It was at this stage in the 
process that a professional writer and a professional layout artist joined the self-study process. 

Some of the subcommittee reports were still problematic even during this late stage in the process, and for different 
reasons. Strong personal biases marked some of the reports. Other subcommittee reports were primarily descriptive 
boilerplate text. Some reports failed to grasp the complexity of their issues. The task force, understanding it had to 
tell the story, reconstructed these reports. The exception was for the eight chapters written by college and school 
deans. The task force felt that it was important that these chapters contain more of a personal feel to them. 

While the task force created the message for the story, it was the professional writer who created the tone and 
institutional voice for the story. Working in concert with the professional layout artist, the writer found ways to clearly 
and efficiently communicate major points to the readers. 

Knowing from the beginning that the report would say that the taskforce believed the university is meeting its mission 
objectives and can continue to do so did not prevent the task force from preparing frank discussions of situations 
that were problematic. There is one notable example of how the self-study process helped the administration address 
a longstanding problem. For the greater part of the decade, in spite of corrective actions, a very important service office 
had been unable to provide the quality and breadth of service to the university community that could be expected. 
At every level of the self-study, starting with the data-gathering survey described above, this was identified as a major 
concern of faculty and staff. Even before the final report was assembled, the administration took a major corrective 
action. The service office staff and its functions were moved within the administrative structure. The staff was placed 
under the supervision of another office. The expanded office was directed to develop a one-stop service philosophy 
for its clients. 

The task force prepared an analysis of why, for the first time in its history, the university experienced a major decline 
in its enrollment in spite of all of the resources and advantages it had to continue its heretofore-steady growth in 
enrollment. Resolution of that situation is in a longer time frame, and the issue is being addressed. 



Energizing the Campus 

The process of measuring the degrees of interdependence among units on the campus raised awareness of how units 
could develop "silo vision” while still meeting their units’ mission goals and objectives. Forced interaction between 
leaders of 67 campus units created new dialog that continued for months. The exercise described below could be 
repeated on a regular schedule, with the expectation that the understanding and appreciation of the complex 
workings of the university would lead to a stronger infrastructure. 

The design of the activity to engage the university community, faculty and staff, originated from the mid-level of 
academic administration, the Deans Council. Two different data-gathering exercises were developed. Paper copies 
^f the survey instruments are available. 
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First, a survey was prepared for all faculty, administrators, and appropriate staff members that collected information 
on these issues: 

1. Rate your department’s degree of responsibility for the following elements of UALR’s mission. 

Using a scale of 1-5, faculty responded to the 17 separate parts in the university’s mission statement. 

2. Rate these attributes in terms of how well they advance the department’s mission. 

Using a scale of 1 -5, faculty responded to how well! 0 identified departmental attributes advance the missions 
of the department, college/school, or the university. 

3. Identify the departments where interdependent working relationships exist with your department. 

All 38 academic units and 29 nonacademic service units were listed. Respondents were asked to identify the 
unit that employed them, to list the three units on campus most dependent upon their unit and then the three 
units that their employing unit is most dependent upon. 

4. Rate the degree of interaction your department has with institutions and organizations external to the 
university. 

The degree of interaction was measured on a scale of 1-5. 

5. Comment briefly on the following questions. 

Respondents were asked to identify their unit’s unique contribution to advancing the university’s mission, and also 
to identify units with which their employing unit needs to create better interdependent relationships in order to 
advance the university’s mission. 

Asecond strategy to raise the awareness of interdependence wasdesigned. Unitheadsfrom allacademicand service 
units were randomly paired. They were asked to meet, discuss the questions listed below, and then submit a written 
report to the self-study coordinator. Not even the campus’ chief executive officer was immune from reporting. 

1 . Identify the current goals of your unit. 

2. Which type of working arrangement, interdependent or dependent, best advances these goals? 

3. How well is your unit achieving its goals? 

4. List campus units that are absolutely dependent upon your unit to advance their goals. 

5. Describe three examples where an appreciation of interdependence between your unit and other units helped 
advance your unit’s goals. 

6. Describe three examples where insufficient appreciation of interdependence prevented your unit from 
meeting its goals. Describe what was done to impress upon the other unit how dependent your unit is on the 
other unit. 

As mentioned above, so many responders identified one service office in their responses to question number 6 that 
the office was merged with another and put under different leadership before the team visited the university. 

The second activity was done during the period of time when the 17 subcommittees were writing their reports. 



Richard H. Hanson is Dean of the Graduate School at the University of Arkansas at Little Rock. 
Sally C. Crisp is Instructor of Rhetoric and Writing at the University of Arkansas at Little Rock. 
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Table 1. University of Arkansas at Little Rock 



Representative Questions and the Responses Regarding 
Missions and Interconnectivity on the Survey Administered to the 
Faculty and Some Members of the Staff 



Faculty and staff recorded their response on a scale of 1-5. 

1 = Little or none 5 = Great NA = Not applicable 



1. Rate your department's degree of responsibility for the following element of UALR's Mission. 

Example of a Shared Value 

To develop student intellect. 

Prufessiohai Studies 
Science and Engineering 
Law 

Education 

Arts and Humanities 
Business 

Example of Item Validity 

To serve/strengthen society by enhancing awareness in scientific arenas. 



Science and Engineering 4.43 

Professional Studies 3.33 

Education 3.16 

Business 2.70 

Arts and Humanities 2.37 

Law 1.39 

Example of College Differences 

To promote humane sensitivities. 

Education 4.52 

Law 4.50 

Professional Studies 4.47 

Arts and Humanities 4.45 

Science and Engineering 3.31 

Business 2.70 



2. Rate these attributes in terms of how well they advance the department's mission. 

Example of Incongruity 

My academic research and creative endeavors advance the department's mission. 



Law 


4.50 


Professional Studies 


4.19 


Arts and Humanities 


4.18 


Science and Engineering 


3.91 


Education 


3.72 


Business 


3.48 



4.72 

4.72 

4.67 

4.64 

4.58 

4.57 



3. Identify the departments where interdependent working relationships exist with your department. 

4. Rate the degree of interaction your department has with institutions and organizations external to the university. 

Example of a Talking Point 

What degree of interaction does your department have with pre- K-2 public/private schools? 



Education 


3.20 


Arts and Humanities 


3.07 


Science and Engineering 


3.06 


Professional Studies 


2.19 


Business 


1.35 


Law 


1.33 
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How Do You Spell Self-Study Relief? 

R-O-L-A-l-D-S! 

One Dose Will Last for Up to Ten Years 



John Loucks 
Marilyn Ewing 
Juneil McQueen 
Ann Judd 

Despite the positive benefits of a self-study, the process adds stress to the work environment. The project is difficult 
and must be accomplished while administrators, faculty, and staff are fulfilling the responsibilities of an already 
demanding workload. At Seward County Community College CSCCC), we took these negatives into account while still 
in the planning stages. From the inception of the process in October 1997 to its completion in March 2000, we 
purposefully accentuated the positive by emphasizing that the self-study gave our campus community the 
opportunity to celebrate the advances of the previous ten years. Maintaining the celebratory attitude through 
committee meetings, missed deadlines, corrupted computer files, and the unexpected departure of key committee 
members was sometimes difficult, but the steering committee managed to diffuse the stress with humor, excellent 
planning, and social “breathers.” As with all medication, we discovered that our efforts had the unintended side effect 
of improving morale across the campus. 

Our Self-Study Coordinators, the editor, and the resource room coordinator worked to alleviate stress by following 
the steps alluded to in the titular acronym. Each of these persons will relate his or her role in each step. 

□ Recruit ^ 

It’s very important to recruit coordinators who have the respect of the campus and the organizational skills to keep 
the process on track. The coordinators must also be mediators, peacekeepers, motivators, and friends. The editor 
should be someone who will stay on task and avoid becoming too attached to the words she/he has written. The 
resource room coordinator should be an excellent organizer and be willing to be the data keeper to ensure 
conservation of the patterns of evidence. Those recruited for this position must not be door-slammers, foot- 
stompers, or voice-raisers; the key characteristic they must possess is patience, which will be continually tested 
throughout the process. 

□ Organize 

Our care in developirig a two-and-one-half year plan kept the committees on track despite the various calamities 
that inevitably occur in long-term projects. Included in the discussion of organization, however, will be the warning 
that the finalized self-study rarely follows the original plan. The report will likely be organized more in accordance 
with the information gathered than with the original outline. Steering committee members must be flexible and 
resourceful enough to reorganize when conditions require it-and to smile all the while. 

Of course, the process must be organized with timetables and due dates. Time must be allowed for steering 
committee members to familiarize themselves with the requirements of the report. Subcommittee chairs should 
be chosen with regard to their reputations for diligence. Handouts will include a narration of the process used to 
appoint subcommittee chairs and key organizational materials used to manage the project 

The team visit requires another round of organization. Our resource room chair organized the resource room by 
anticipating the needs of the team, from aesthetics to rapid access to documentation. She will provide a narration 
of the steps she took to accomplish her goals, as well as a diagram of the room's arrangement. We will also discuss 
the special accommodations we provided for the team to make each member feel welcome. 
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□ Laugh 

At SCCC, steering committee members were given their assignments at a dinner meeting. One of the coordinators 
cleverly gave each member his/her charge using the words of the tape-recorded voice in Mission Impossible. 
Clever treats were brought to each steering committee meeting. To assure careful reading, the editor included 
humorous text in the pages she submitted for approval by the steering committee (the humor was very carefully 
deleted after printing the draft to avoid inadvertent inclusion in the final report}. The handout will include some 
of these passages. 

□ Appreciate 

All steering committee members remained positive during the process, purposefully accepting the subcommittee 
reports with applause and thanks. Though many reports needed more work upon first submission, steering 
committee members asked for those changes in private. Many subcommittee members were classified staff 
terrified of writing, so the editor empowered them by reminding them of their expertise and the special point of 
view that their role on campus afforded them. She emphasized that the writing was her job and that she would 
not judge others by their writing skills as long as they did not makfi fi in of her for her housekeeping or typing skills. 
Although some of the reports were more descriptive than evaluative, they provided the information necessary to 
draw important conclusions about the criterion being covered. A caution should be noted here: Do not provide 
subcommittees with too many self-studies as examples. The editor caught some items taken word for word from 
self-studies kept in our library for subcommittee use. We must remember that some staff members may be unaware 
of the definition of plagiarism. 

One important means of showing appreciation for the hard work of the entire staff was a special convocation held 
to award each committee member with a copy of the Self-Study Report. Each committee was called to the stage 
en masse: then the president and the Self-Study Coordinators thanked each committee member individually. Gifts 
were presented to the editor and the coordinators at all-staff meetings. The most effective signs of appreciation, 
however, were literal pats on the back and kind words. 

□ Individualize 

Although representatives of most colleges pick up copies of successful reports at the NCA Annual Meeting, they 
should be used only as guides. Each institution has its own history and emphases, and these unique elements 
should guide the writing of the report. In our Self-Study Report, we introduce chapters with college legends or a 
discussion of a quality or event unique to our institution, lending individuality and interest to the report. In addition, 
the organization of the report was different from any we viewed. For instance, it was more logical for us to include 
assessment in two chapters, keeping it close to its relevant usage, rather than making assessment its own chapter. 
We encourage others to take this organic view of organization. Our chapter division pages contained testimonials 
from and pictures of our students. These and other creative approaches generated needed energy and enthusiasm 
when the enormity of the process was most daunting. 

□ Delegate 

Delegation does not end with the appointment of committees. There are times when steering committee members 
can be overwhelmed with immediate responsibilities, and it is important that the collegiality of the steering 
committee is established enough that a member can yell “Help!” when in fear of drowning. All members of the 
committee should be sensitive to the possibility that one person may be overloaded and offer that help even if it 
is not requested. Finally, the key to delegation is to trust those to whom one delegates; otherwise, no aid can be 
given. 

□ Soiree 

We don’t want to break parallelism with the end of our acronym, so we ask our readers to think of soiree as a verb, 
despite the likely objection of lexicographers. We celebrated early and often during the self-study process, 
beginning with a dinner at the local country club for newly-appointed steering committee members, and ending 
with a food fest following the visiting team’s departure. Because each steering committee member attended the 
NCA Annual Meeting at least once, celebrations included gourmet dining in Chicago, especially memorable for 
those of us who rarely have the opportunity to leave the Liberal, Kansas, city limits, where our options are limited 
to greasy spoons. A complete description of our celebrations will be included in the handout. 



Committee members will share advice during the last portion of the presentation, most of which comes from the clarity 
of hindsight. Although we are proud of the outcome of the NCA accreditation process, we know that we should have 
°pent more time preparing our students for the visit. Among the topics we will discuss is the unfortunate result of not 
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having clearly defined the word assessments our students in preparation for the visit. We will discuss the time that 
can be wasted worrying about capitalization and pronoun use. We will exhibit the result of a faulty printing job and 
propose steps to take to avoid last minute jitters, but these setbacks and quandaries are nothing that a little ROLAIDS 
won't cure. 



John Loucks is Philosophy and History Instructor and Chair of the Humanities Division at Seward County Community 
College in Liberal, Kansas. 

Marilyn Ewing is Accounting Instructor at Seward County Community College in Liberal, Kansas. 

Juneil McQueen is Office Technology Instructor at Seward County Community College in Liberal, Kansas. 

Ann Judd is Lead English Instructor at Seward County Community College in Liberal, Kansas. 
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Envision Your NCA Visit: 
How Little Things Mean a Lot 



M. Joanne Rathburn 
Lori Weyers 
H. Jeffrey Rafn 

Envisioning or picturing in your mind your NCA visit helps in setting goals for the process. Research, brainstorming, 
and mapping your ideas on paper gets the process started. To meet your goals, you need an overall plan and a focus 
on details. Northeast Wisconsin Technical College (NWTC] in Green Bay, Wisconsin, followed such a process to 
achieve a very successful event in October 2000. In this paper, NWTC will share some of its planning, writing, 
communicating, studying, visualizing, recognizing, and celebrating. 

NWTC is a two-year technical college in the Wisconsin Technical College System. The College has a main campus 
in Green Bay, and other campuses in Marinette and Sturgeon Bay. 



Planning 

The Planning Committee consisting of the college president, administrators, management, and faculty set the pace 
for the process. This committee: 

o Established the manner in which the report would integrate with and support other college initiatives 
o Set timelines 

o Outlined the process to include as many members of the college as possible 
o Decided on the organization of the report and produced the preplan 
o Selected the Steering Team from a list of volunteers 
o Approved a training process and release time for the Steering Team 
o Reviewed the recommendations from the previous visit 
o Contracted for a college-wide climate study 
o Discussed the issues that were anticipated as current concerns. 



Writing 

The Steering Team representing faculty, administration, management, and support staff led the research process 
under the leadership of a faculty coordinator. The Steering Team: 

o Attended a workshop presented by co-chairs of a successful NCA visit at Fox Valley Technical College 
o Selected an area of responsibility for research 
o Recruited college personnel to join in the research 
o Studied the NCA Handbook 

o Wrote questions to be answered based on NCA criteria 

o Researched answers to the questions, collected resources, and wrote a draft of a chapter for the report 
o Reviewed the drafts from each research team 

o Evaluated the information and listed strengths, opportunities for improvement, and activities in progress. 
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The coordinator, a faculty member, was responsible for the overall progress and: 
o Served as a liaison between the Planning Committee and the Steering Team 
o Facilitated the weekly meetings of the Steering Team 
o Revised the chapter drafts 
o Compiled the final report 

o Communicated the progress to all members of the college 
o Planned the Resource Room 
o Served as a contact with the visiting NCA team. 

Communicating 

Communication with all college personnel was a priority in the overall plan. These steps were taken to keep everyone 
informed: 

o Wrote articles in college publications 

o Sent letters and memos to each member of the college, including advisory committee members and adjunct 
faculty 

o Held focus meeting with students 

o Sent memos to all students, which were distributed by faculty in classes 
o Displayed posters throughout all campuses 
o Visited all campuses to meet with faculty and staff 

o Presented updates to divisions or departments at meetings throughout the process 
o Reported on progress at all-college inservices during the process 
o Sent copies of Executive Summary to everyone 
o Sent email to everyone 
o Displayed announcements on TV monitors 

o Sent weekly updates to everyone during the month preceding the visit 
o Planned celebration activities for each all-college inservice 
o Presented information at Board of Trustees’ meetings. 



Studying 

Of course, the teams researched NWTC thoroughly. In addition, the teams: 
o Studied the NCA Handbook 
o Read reports from many other colleges 
o Attended state conferences 
o Attended the NCA Annual Meeting 

o Spoke with others who had completed the visit or were in planning stages 
o Questioned NCA evaluators 
o Contacted the NCA liaison. 



Visualizing 

Visualizing was an important activity in the preparation for NCA. Team members visualized the format of the report 
in an initial step. The Resource Room team visualized the organization of the room, the order of the resources, and 
the amenities for the visit long before the visit occurred. (The coordinator even visualized the happy report-out 
session!) 
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Recognizing 

Everyone worked hard and put in many extra hours. Recognizing those efforts was a goal. The following list of activities 
includes different kinds of recognition: 

o Introducing Steering Team members at an inservice 
o Publishing the names of all teams in the college newsletter 

o Posting the names of all team members on a major bulletin board for the duration of the process 
o Presenting special college mugs to all team members 
o Sending bookmarks to everyone as a reminder of the dates of the visit 
o Sending copies of the complete Self-Study Report to all who requested them 
o Paying for team members to attend NCA Annual Meetings. 



Celebrating 

The Planning Team wanted this process to excite, energize, and motivate the college for continued performance. The 
theme, “NWTC Celebrates Learning with NCA,” set the tone for the whole process. The teams emphasized learning 
and celebrating what we do through community, cooperation, and camaraderie. Some fun activities were: 

o Sing-a-long to introduce NCA to everyone at an all-college inservice [verses from popular songs were 
rewritten for our purposes] 

o Notepads with the message “NWTC Celebrates NCA in 2000-01” given to everyone 
o Job Fair to introduce personnel to the different teams 
o Special edition newspaper with “Help Wanted” ads to interest volunteers 

o NCA Challenge, with a game show format and competing teams answering questions about NCA and NWTC 
[everyone in the audience received the questions and the answers!] 

o Prizes awarded to participants in all events 
o Ice cream social to work on assessment project 

o An all-college party funded by the bargaining units and the college after the visit. 

Conclusion 

In summary, many little and not-so-little efforts led to NWTC’s continued accreditation with the next comprehensive 
evaluation in ten years. The efforts that stand out in reflecting on the whole process are these: 

o The support and active involvement of the administration and the Board 
o The active involvement of faculty and staff 

o The special attention given to the concerns from the previous review 
o The additional chapter in the Self-Study Report that summarized the evaluation process 
o the continual effort of communicating the events to everyone 
o The fun and enthusiasm everyone had. 



M. Joanne Rathburn is Communication faculty member at the Northeast Wisconsin Technical College, Green Bay ; 
Wisconsin. 

Lori Weyers is Vice President for Learning at Northeast Wisconsin Technical College, Green Bay, Wisconsin. 

H. Jeffrey Rafn is President of the Northeast Wisconsin Technical College, Green Bay, Wisconsin. 
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Why the NCA Accreditation Process 
Facilitates Strategic Visioning: 
When NCA Conies, 

It's Too Late to Phone a Friend 



John A. Ludrick 
Leslie D. Cralj 



Introduction 

This session explores the process of going through a special emphasis self-study step by step. Southwestern 
Oklahoma State University utilized the self-study process to engage in strategic visioning. Southwestern, like many 
universities, faces major impact from technological and political-eponomic influences. These influences exist 
externally as well as internally. Changing demographics, fluctuating enrollments, uncertain funding levels, and 
information-age technology affect all major aspects of the institution. Southwestern needs to be in a position to do 
more than simply cope with these impacts. It must enable itself to use these influences to its advantage. Good 
decisions based on sound data analyses will result in Southwestern Oklahoma State University’s becoming an even 
stronger and more progressive institution in the future. 



Strategic Visioning 

The session will provide insights into the joys and tribulations of developing our strategic visioning plan, the “Agenda 
for Excellence.” This publication characterizes a plan for the university for five years into the future as the major goals 
we will strive to achieve. These goals, as developed by a large group of internal and external stakeholders, reflect the 
driving forces and comparative advantages of Southwestern Oklahoma State University. Examples of the Strategic 
Goals and Initiatives are: 




By 2003: Southwestern will be serving 5500 students and will have advanced its position as one of the premier 
universities in Oklahoma with an emphasis on quality education. (Current enrollment of Southwestern 
is 4915.) 

By 2001: The university will have an effective marketing plan that promotes greater exposure of its departments 
and programs. 

By 2001: The university will have enhanced the effective and efficient management of its resources. 

By 2003: The university will have established cooperative arrangements to enhance its mission of teaching, 
scholarly activity, public service, and economic development. 

By 2005: The assets of the Southwestern Oklahoma State University Foundation will have increased to $10 million. 
(Currently the balance is approximately $6.5 million.) 



.. * 
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The Self-Study 

Planning and goal-setting permeate various levels of most institutions; rarely does the process become all 
encompassing so as to achieve a cohesive unit that involves the entire university. The NCA accreditation process, 
especially the special emphasis type of self-study, does afford just such an opportunity. While faculty members may 
be very adept with the strengths and problems of their individual units, seldom are they asked to undertake the 
visioning involved with NCA accreditation for an entire university. An essential strength is that individuals who work 
with the NCA process focus on a university level rather than at a departmental level. 



The Process 

When Southwestern embarked on the NCA accreditation process and the steering committee was formed, one 
common goal was determined: “coming up with a strategic vision for Southwestern Oklahoma State University that 
would drive the institution into the future.” The steering committee wanted to provide administrators, faculty, students, 
and other stakeholders with a comprehensive agenda that would build on the strengths that currently existed within 
ihe university, it was determined that the end product would be a document that would not just sit on someone’s 
bookshelf but that rather would have a significant impact on the future of the institution. The steering committee was 
committed to the future of the university and the visioning efforts. Also paramount was a desire to look for 
opportunities on which the university could capitalize. A natural part of the process was to try to identify and address 
potential problems or concerns that might exist. 

The rationale for this presentation is to share with others the sequence that the university went through over a three- 
year period during the NCA accreditation process. The hope is that some insight will be extended into the potential 
rewards of such a process as well as point out some of the potential pitfalls encountered and that others are likely 
to encounter. The presenters will emphasize the ability to utilize the accreditation process for institutional 
improvement and excellence. 

Institutions that are on the verge of embarking on the self-study process, especially those that are contemplating 
using strategic visioning for their type of special emphasis self-study, will benefit by attending the presentation. 
Forward-thinking institutions that want to achieve objectives through the more efficient utilization of their limited 
resources, that are currently engaged in this type of self-study, and that are in the final stages prior to the evaluator’s 
visit can benefit from this presentation. Activities and tasks will be highlighted from a practical standpoint. 



The Steps in the Process 

The presentation is a comprehensive, in-depth look at the planning process recently completed at Southwestern 
Oklahoma State University. This will be accomplished through a PowerPoint slide presentation that will show the 
timelines and tasks involved in arriving at and subsequently implementing the “Agenda for Excellence.” This process 
involved all levels of the university, including students, faculty, administrators, and other university stakeholders. The 
broad categorical outline of the Southwestern Oklahoma State University plan is as follows: 

1. Data collecting 

2. Data assimilation 

3. Developing merged scenarios 

4. Consensus-building 

5. Writing the self-study 

6. Getting reaccreditation 

7. Implementing initiatives for goal achievement 

The president of the university formed a Planning and Resource Council to serve as an oversight committee 
throughout the process. The Council was also charged with giving advice. Next, scan teams were formed to research 
the perceptions of internal and external stakeholders and individual values that the university holds dear. Internal 
stakeholders consisted of students, faculty, staff, and administrators. The external scan team looked at perceptions 
of parents, employers, government officials, alumni, and friends of the university. The values scan team made an effort 
O 
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to gain insight into the values that people considered important within the university. These results helped determine 
what were perceived to be the university’s strengths, weaknesses, and opportunities. Once these data were compiled, 
the scan teams wrote reports of their findings. 

The university then conducted a massive two-day meeting that was attended by students, faculty, administrators, and 
community stakeholders. The purpose of this meeting was to identify the goals for the university. We were advised 
to keep the list of goals very specific and small in number. Ten goals were initially identified and they were narrowed 
down to five from which the "Agenda for Excellence” was crafted. Action teams were then formed with faculty, staff, 
administration, and student representatives. Each action team was charged with drafting plans to achieve each of 
these goals. The action teams identified persons with primary responsibility for implementing the various actions and 
produced timelines for achievement. The university is currently engaged in this process. 

The members of the NCA steering committee were involved at all phases of the aforementioned visioning process. 
Each member chaired a work team to research and write the Self-Study Report and address the different criteria for 
accreditation. The final document contains the results of the visioning process and the reports from the various action 
teams on how to achieve the goals. 



Conclusion 

This session focuses on our recent self-study experience that began in the fall of 1 997 and culminated with the site 
visit in October 2000. The strategic visioning engaged in by the university and the process of the NCA self-study greatly 
benefited Southwestern Oklahoma State University. The conclusion is founded on the university’s establishing goals 
for the future. It made it possible to address change, examine ourselves to find our comparative advantages, and 
identify driving forces that we expect will shape our future. 

We want to share our experiences with those university representatives who are presently in any phase of the 
accrediting process. Participants will be actively engaged in the presentation through the utilization of a discussion 
and question-type format. 

Strategic visioning is vital in higher education as universities and colleges fight ever-increasing battles for funding 
and the public demands greater accountability for the use of their tax dollars. In a rapidly changing environment, the 
NCA accreditation process, specifically the special emphasis self-study, allows us to have dynamic flexibility to adapt 
and change to meet demands 

During the session, the presenters will reveal a chronological sequence that will emphasize the vigorous planning 
and dedication of everyone involved in order to accomplish the examination of the present status of the university. 



John A. Ludrick is a Professor and Assistant to the Dean in the School of Education at Southwestern Oklahoma State 
University in Weatherford. 

Leslie D. Crall is an Associate Professor and Chair of the Business Administration Department in the School of Business 
at Southwestern Oklahoma State University in Weatherford. 
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Southwestern Oklahoma State University 
Figure 1. Initial Planning Process 
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Southwestern Oklahoma State University 
Figure 2. Annual Planning Process 
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Centralized Data Collection: 
A Strategy for Building a 
Data-Rich Resource Room in a 
Data-Poor Institution 



Liz Donnelly 
Lloyd Musselman 



Institutional Overview and Context 

Oklahoma City University is a private, comprehensive, United Methodist-related university offering nationally 
acclaimed programs in dance, law, and music; one of four youth ministry programs in the country; and the only 
international business degree program in Oklahoma. OCU ranks among the nation’s best universities in The National 
College Review Guide, U.S. News and World Report, The Student Guide to America's Best 100 College Buys and 
America's Best Christian Colleges. OCU students receive practical, broad-based education, coupled with an emphasis 
on their ethical and religious potential, in an environment structured to maintain the United Methodist tradition of 
education without indoctrination. The student/faculty ratio is 14:1, the best in Oklahoma. 

OCU boasts a highly diverse student population, with students from 48 states and more than 60 countries. 
Approximately 2000 of the 4000 enrolled are undergraduates. Academic programs are offered by the Petree College 
of Arts and Sciences, Meinders School of Business, School of American Dance and Arts Management, School of 
Music, Kramer School of Nursing, Wimberly School of Religion and Graduate Theological Center, and the School of 
Law. The campus is located in the heart of Oklahoma City, offering students a wide range of services and cultural 
opportunities. 

The current president, Dr. Stephen Jennings, began his service to the university in 1998. Under his leadership, the 
institution has focused on quality-jmprovement initiatives through data-driven decision making. This direction has 
led the campus to critically evaluate all aspects of its information management systems. The desired outcomes of 
these efforts are to improve services to students and the quality and consistency of institutional data and information. 



Integrating Data-Driven Management with the Self-Study 

In August of 1 999, Oklahoma City University began the process of developing a plan for implementing a self-study 
in preparation for a North Central Association visit in February 2002. Inherent to this process is the collection of 
information, resources, and data from variety of administrative units for the purpose of evaluating the programs and 
services offered at OCU. The Steering Committee quickly recognized that the collection of institutional data and 
informational resources posed a significant challenge because many of the administrative and academic units did 
not and do not systematically collect and analyze data as a matter of practice. 

Nevertheless, it was recognized that the self-study process presented an opportunity to move the campus away from 
underutilizing data and resources to a more data-rich culture. The looming North Central visit required us to build 
and organize a core of historical data, and identify data “holes” where essential missing data could be mined. This 
effort produced a comprehensive list of institutional resources and a central location where stored data are accessible 
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to all members of the Self-Study Steering Committee. The Steering Committee determined that the collected data 
must not only assist in researching the self-study, but must also be instrumental in systematically maintaining 
up-to-date data and information long after the completion of the NCA visit. This information must be instrumental 
in future long-range strategic planning initiatives and data collection efforts. 



Issues and Problems 

Many issues emerged in the process of determining just how to collect and store the necessary data. The first 
concerned enhancing the consistency and reliability of data in existence at the time of the initiation of the self-study 
planning. The data reported internally and externally were often inconsistent, varying according to the reporting 
administrative unit. For example, the freshman retention rate was reported in several different ways due to the absence 
of a shared definition of what precisely comprised the cohort of students to be measured. 

Another issued involved the low level of willingness of several administrative units to provide data to institutional 
members outside their area. This issue was compounded by a variety of factors. Many administrators were hesitant 
to provide data due to a lack of confidence in the accuracy of the data and lack of understanding of how to collect, 
analyze, and present data. The situation was exacerbated by the lack of administrative time and resources many units 
allotted to the systematic collection of data in each administrative unit. Data collection for the purpose of data-driven 
decision making was not a priority in many units because it was not associated with institutional rewards or 
consequences. In addition, limited human resources, inadequate training in information systems, and scarce fiscal 
resources lowered the priority that units gave to collecting and reporting data. 

Given these issues, the Steering Committee was sensitive to the need for a palatable, cost-effective, low-threat 
method of institution-wide data collection. Therefore, the Steering Committee devised a protocol of centralized data 
collection. The organizing body of this initiative was called the Data Collection Team and served as a subcommittee 
of the Steering Committee. Members of the Data Collection Team included the Director of Information Technology, 
Director of Institutional Research, Director of Overseas Programs, and Vice President of Student Development. 



The Process of Centralized Data Collection 

A brief word about the organizational structure of the Steering Committee is appropriate. The Steering Committee 
included five subgroups called Criterion Teams, each responsible for one of the five Criteria for Accreditation. One 
of the first tasks assigned to the Criterion Teams was to identify the data each believed would be needed to begin 
working on their portion of the self-study and to submit requests for data to the Data Collection Team. 

Each Criterion Team developed a wish list of data, reports, and documents they anticipated would be useful in 
researching and writing their portion of the self-study. The data collection protocol incorporated a review of all 
requests for data by members of the Data Collection Team. The Director of Institutional Research managed the f|ow 
of requests and the data subsequently received, This strategy served as a type of triage, consolidating two or more 
similar requests for data into one and funneljng requests to the person on campus most likely to have access to the 
data requested. This process prevented one person pn campus from receiving multiple requests for similar data and 
from receiving requests for data for which they were not responsible. 

As the Criterion Teams progressed through the self-study process, new data needs emerged and were forwarded to 
the Data Collection Team. The Data Collection Team continuously evaluated and distributed the data requests in an 
organized fashion. 

Triaging requests narrowed the collection of data, reducing workloads and creating a tight, consistent corpus of data 
to be incorporated into the Self-Study Report. Senior officers were willing to cooperate because they were confident 
that requests had been subjected to a prescreening process and were legitimate. A member of the library staff 
cataloged the data entering the Resource Room, maintaining an accurate and up-to-date record of the holdings. The 
updated list of resources was regularly emailed to members of the Steering Committee. 



0 
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The mass of collected data and information will essentially make up the contents of the Resource Room. At the time 
of this writing, Oklahoma City University is in the middle of its self-study process. Criterion Teams are submitting the 
first draft of their reports to the Steering Committee. Inherent in the process of reviewing the submitted draft will be 
suggestions to the Criterion Teams from the Steering Committee for supplemental data to strengthen the reports anc) 
support the recommendations made in the reports. Supplemental data gathered by the Criterion Teams will be 
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forwarded to the Data Collection team to be cataloged and included in the Resource Room, either as hard copy or 
as a list of resources and their location. 



Lessons Learned 

Several obstacles emerged in implementing this process. The Data Collection Team quickly learned that requests for 
data had to be extremely clear and comprehensive. Written requests were often followed by a telephone call from a 
member of the Data Collection Team to ensure that the request was understood. Requests for data should include 
a detailed explanation of the data collected, the number of years back the provided data should include, and the format 
for the data. 

After a period of trial and error, requests for data included due dates. It was very helpful to forward an email reminding 
providers of the data of the impending due date about one week prior to the due date. Telephone contacts were made 
to inquire about data requests not received by the due date. 

Timing due dates for returning data was tricky. The Data Collection team was sensitive to the need to provide data 
providers a reasonable span of time to submit data; however, lengthy due dates would unnecessarily prolong the 
process and prevent the Criterion Teams from moving forward on their work. Each data request was analyzed as to 
the availability of the data requested and the workload of the office or person to whom the request was made at that 
time in the academic calendar. Turnaround times that were too short irritated data providers and eroded the credibility 
of the process. 

Handling and organizing data requests was very time-consuming. The person or persons assigned to this task require 
clerical support and should have good computer skills. In our case, the Director of Institutional Research maintained 
records of requests, including when the request was made and when it was due, on an Excel spreadsheet. This proved 
useful, but it did require more time than the Data Collection Team originally anticipated. 



Post Self-Study 

Upon completion of the self-study process, the Resource Room will be provided a yet to-be-named permanent home. 
It will be the responsibility of the Office of Institutional Research to annually update and supplement the holdings and 
forward appropriate items to university archives. The mass of data will be useful for future strategic planning initiatives 
and institutional research projects. 



Conclusion 

This strategy has strong possibilities for institutional improvement. This collection of data will serve as the basis and 
guide for future data collection and will provide historical context for strategic planning. Through all of this, the 
Steering Committee hopes to alter the culture of the campus by making data-driven decisions reflexive rather than 
unusual or even agonizing. In addition, the comprehensive list of available data will facilitate future data-collection 
efforts, providing records foryear-to-year comparisons of institutional information such as enrollment trends and data 
necessary for evaluating and assessing quality-improvement initiatives. 



Liz Donnelly is the Director of Student Academic Support Programs at Oklahoma City University. 

Lloyd Musselman is Professor of History and Chair of the North Central Association Self-Study for Oklahoma City 
University. 
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Electronic Data Gathering in the 
Preparation of the Self-Study 



Brenda LeMaster 
Lawrence Johnson 

The University of Cincinnati (UC) went through reaccreditation in the spring of 1999. A self-study steering committee 
was appointed in January 1997, and a decision was made soon after to pursue the special emphasis option for our 
self-study. It was the intent of the institution to use the reaccreditation process as a way to measure the effectiveness 
of ongoing institutional priorities in the areas of pedagogy, research, globalization, interdisciplinarity, and technology. 

One of our first challenges was from the staff at NCA, who felt that five special emphases were too many, and we 
needed to find a way to narrow our focus. After much discussion we selected pedagogy and research as our special 
emphases, with the understanding that we would look at each of these areas through the lens of globalization, 
interdisciplinarity, and technology. This proposal was accepted by the NCA. 

UC is a large, complex institution with multiple missions. Our sixteen colleges include an open access college on the 
main campus, a medical school, and a music conservatory. Over the years, our colleges developed cultures of 
autonomy that resulted in many internal barriers to institutionally driven initiatives. A very successful, centrally 
administered faculty development program, along with support for several pedagogical initiatives over the past 
several years, had begun to erode some of those barriers. However, preparation for the self-study required that we 
find a way to obtain active participation from all of our academic and administrative units. 

In this session, we will discuss the steps we took to organize the work of the self-study, including hiring an internal 
consultant to oversee the process. One of the most effective aspects of our process was the development of templates 
for collection of data. We will discuss the development and use of the templates, which we feel are useful tools 
regardless of the size of your institution. The following is the template for our special emphases. 




NCA Special Emphases Template 

Directions 

Please provide on a disk short narrative statements that adequately supply the information requested below. 

The information you submit will be integrated into the Self-Study Report for the University. 

a) Enter your response after each sub-heading as indicated. 

b) Supporting materials and documents for narrative statements should be included in the 
appendices. Please label supporting materials/documents with the sub-heading to which they 
are related (example: IB2: Interfacing Technology and Pedagogy-Outcomes). 

c) Not every heading may apply to your unit. If you have no information related to a given heading, 
please indicate that it is not applicable to you. 

d) After answering all sub-items for a given criterion, please provide an assessment of your unit 
related to the criterion as a whole. In this assessment, describe how adequately your unit 
currently meets the criterion and any plans your unit has for the future regarding the criterion. 
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I Pedagogy 

A. Interdisciplinary Pedagogy 

1. Current Initiatives: Describe activities in your unit that support interdisciplinary pedagogy. 
Response: 

2. Outcomes: Describe the outcomes of the initiatives and/or data that demonstrate the impact of 
these activities on student learning. 

Response: 

3. Improve Practice: Provide documentation that the information in item 2 has been incorporated 
to improve practice. 

Response: 

B. Interfacing Technology and Pedagogy 

1 . Current Initiatives: Describe activities in your unit that support the interfacing of pedagogy and 
technology. 

Response: 

2. Outcomes: Describe the outcomes of the initiatives and/or data that demonstrate the impact of 
these activities on student learning. 

Response: 

3. Improve Practice: Provide documentation that the information in item 2 has been incorporated 
to improve practice. 

Response: 

C. Global Pedagogy 

1. Current Initiatives: Describe activities in your unit that support global pedagogy. 

Response: 

2. Outcomes: Describe the outcomes of the initiatives and/or data that demonstrate the impact of 
these activities on student learning. 

Response: 

3. Improve Practice: Provide documentation that the information in item 2 has been incorporated 
to improve practice. 

Response: 

D. Summary Analysis: In answering the following four questions, provide narratives that relate and tie 
together your discussions about interdisciplinary pedagogy, interfacing technology and pedagogy, 
and global pedagogy. 

1. Strengths: Analyze the strengths of your unit regarding pedagogy. 

Response: 

2. Challenges: Discuss the issues about pedagogy you see as challenging for your unit. 
Response: 

0 
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3. Emerging Issues: Describe emerging issues related to pedagogy. 

Response: 

4. Planned Initiatives: Include a brief description of any additional initiatives you plan to undertake. 
Response: 

II Research/Scholarly Work/Creative Activity 

A- Interdisciplinary Research/Scholarly Work/Creative Activity 

1. Current Initiatives: Describe activities in your unit that support interdisciplinary research/ 
scholarly work/creative activity. 

Response: 

2. Outcomes: Describe the outcomes of the initiatives and/or data that demonstrate the impact of 
these activities on interdisciplinary research/scholarly work/creative activity. 

Response: 

3. Improve Practice: Provide documentation that the information in item 2 has been incorporated 
to improve practice. 

Response: 

R. Interfacing Technology and Research/Scholarly Work/Creative Activity 

1 . Current Initiatives: Describe activities in your unit that support the interfacing of technology and 
research/scholarly work/creative activity. 

Response: 

2. Outcomes: Describe the outcomes of the initiatives and/or data that demonstrate the impact of 
these activities on technology and research/scholarly work/creative activity. 

Response. 

3. Improve Practice: Provide documentation that the information in item 2 has been incorporated 
to improve practice. 

Response: 

C. Global Research/Scholarly Work/Creative Activity 

1 . Current Initiatives: Describe activities in your unit that support global research/scholarly work/ 
creative activity. 

Response: 

2. Outcomes: Describe the outcomes of the initiatives and/or data that demonstrate the impact of 
these activities on global research/scholarly work/creative activity. 

Response: 

3. Improve Practice: Provide documentation that the information in item 2 has been incorporated 
to improve practice. 

Response: 
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D. Summary Analysis: In answering the following four questions, provide narrative that relate and tie 
together your discussions about research/scholarly work/creative activity and interdisciplinarity, 
technology and globalization. 

1. Strengths: Analyze the strengths of your unit regarding research/scholarly work/creative 
activity. 

Response: 

2. Challenges: Discuss the issues about research/scholarly work/creative activity that you see as 
challenging for your unit. 

Response: 

3. Emerging Issues: Describe emerging issues related to research/scholarly work/creative activity. 
Response: 

4. Planned Initiatives: Include a brief description of any additional initiativesyou plan to undertake. 
Response: 

III Appendices 

Attach any data or resource documents that support the narrative provided in items I and II. 



In our session we will share the templates for the five criteria and discuss the twelve basic steps of our self-study. 
Copies of our Self-Study Report are available in the Annual Meeting Resource Room, and provide evidence of the 
effectiveness of our data gathering method. 



Brenda LeMaster is Vice Provost for Institutional Effectiveness at the University of Cincinnati in Ohio. 
Lawrence Johnson is Dean of the College of Education at the University of Cincinnati in Ohio. 
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Focusing on the Focused Visit: 
Strategies for Communication and 
Campus Involvement 



Donald F. Larsson 
Susan Coultrap-McQuin 

Widespread campus involvement is a key factor for any successful self-study. It can sometimes be difficult to involve 
all the necessary campus constituencies even as part of the regular reaccreditation process. The need for campus 
involvement, though, can be even more acute when the self-study is being conducted for a focused visit, where the 
issues and stakes involved in the self-study and visit may be less apparent to the campus community. Effective 
communication can help ensure that the issues for the focused visit are being addressed and that both individual 
groups and the campus at large are involved in the self-study process. 



Background to Our Focused Visit 

In the spring of 1996, Minnesota State University, Mankato (then known as Mankato State University) was visited by 
an NCA Evaluation Team as part of its regular reaccreditation process. The team recommended continued accreditation 
forthe university but also stipulated the need fora Focused Visit in spring 2000. The Evaluation Team highlighted three 
areas needing the attention of a Focused Visit: governance, evaluation and planning, and faculty/educational 
programs. Recognizing the importance of making progress in these areas, the administration and various faculty 
groups developed strategies for addressing the concerns. In the year preceding the scheduled Focused Visit, the 
senior vice president asked us (a dean and a faculty member with nearly two decades of experience on campus) to 
be co-chairs of the visit, including co-authoring the self-study and co-chairing an Advisory Committee with 
representatives from the faculty, support staff, students, and other key groups. 

We approached writing the self-study like any research project that necessitated collecting relevant information, 
analyzing it, and then presenting it clearly to the readers. Not only did we plan to talk to a lot of people in order to 
collect information, but we also wanted to learn as much as we could by reading annual reports from various units, 
minutes of committees, and other documentation related to the focus areas. We understood our responsibilities to 
include extensive communication with groups across campus to make them aware of the visit and of the issues 
involved. We wanted to accurately represent campus views about our progress (or lack of it) and to set the stage for 
a successful visit, which was scheduled for February 2000. 



Identifying Issues and Constituencies 



One of our first tasks was to understand the issues that had been targeted forthe Focused Visit by the NCA Evaluation 
Team. We carefully read the original team’s full report in order to analyze their concerns in the three major areas and 
to be able to explain those concerns to others as the campus prepared for the visit. We found that each of the areas 
included several subareas of concern. The third focus area, faculty/educational programs, for example, was in fact 
quite wide-ranging, including concerns about general education, assessment of student outcomes, criteria for 
promotion and tenure, and faculty development. Equipped with our understanding of the concerns, we were ready 
to begin discussions with members of the campus about our progress. 




Having identified the major issues and concerns to be addressed, we developed a list of individuals and groups on 
campus most centrally involved in work related to governance, evaluation and planning, and faculty/educational 
programs. While the focus areas were primarily related to academic and faculty issues, some (especially the focus 
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on planning) also affected MSI) students, student support services, fiscal affairs, and campus operations. Therefore, 
while most of those consulted were from the academic affairs side of campus, we were careful to encourage all groups 
across campus to participate in discussions and to be knowledgeable about the issues. 

Among the principal people and groups we identified for extended consultation were: 

o University president 

o President’s cabinet 

o Senior vice president/vice president for academic affairs 
o Academic Affairs Council (Including college deans) 
o Executive Committee of the campus faculty union 
o Executive Committees of other bargaining units 
o Department chairs and departments 
o Student government leaders 
o Student Affairs Council 
o Planning Committee 
o Budget Committee 
o MSU Foundation 

o Undergraduate Curriculum and Programs Committee 
o General Education Subcommittee 
o Program Review and Assessment Committee 
o Graduate Committee 
o Learning and Technology Roundtable 
o Faculty Development Committee 

o Faculty Development Coordinator 

o NCA Focused Visit Advisory Committee 

Gathering Information and Feedback 

We collected information through discussions in various meetings and from responses in written formats. We 
envisioned a self-study that would be of use to the Focused Visit Team by [1) providing background information and 
explaining the institutional context for each of the three areas; (2) demonstrating progress (or explaining the lack of 
it) in each area; (3) reflecting as accurately as possible general campus views of the concerns; and (4) indicating 
campus intentions for continued progress after the visit. This vision gave us a working structure as we solicited 
information from campus constituencies who would help us to write the report. 

Over the spring and summer of 1999, the focus areas were discussed in Academic Affairs Council, the chairpersons’ 
workshop, Student Affairs Council, each college, various committees, our administration/faculty union council, and 
two open meetings. Some of these discussions took place without our direct presence, and we received reports of 
the groups’ conclusions; however, individually or together, we also met with many of these groups. Some meetings 
were planned to solicit a maximum amount of commentary. For example, when meeting with department chairs and 
in an administration/faculty union retreat, we explained the Focused Visit, briefly outlined the issues, and then broke 
into small groups that returned comments, ideas, and questions on aspects of the Focused Visit. Other meetings were 
more wide-ranging. Our meeting with the Student Affairs Council, for example, solicited open-ended comments in 
the key areas, but the conversation also touched on related topics. Still other meetings were more narrowly focused. 
For instance, we asked the committees most directly related to a particular issue (such as the Planning Committee 
in relation to the second area of concern) to explain what work they had done in their area since the NCA visit, to 
provide any documentation they had, and to explain what work still needed to be done or what other areas needed 
to be addressed. We received suggestions and feedback from regular meetings with the Focused Visit Advisory 
Committee as well. 

We also kept the larger campus community aware of our efforts. In April 1999 we sent a letter to all key groups and 
individuals and to the campus at large, informing them of the reason for the upcoming Focused Visit, giving examples 
of some of the information about progress on the focus issues that we had begun to receive from campus groups, 
and soliciting responses from any interested party. We provided regular updates about our plans, activities, and 
requests for information on the faculty/staff/student email announcements. We created a web site that explained the 
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Focused Visit, outlined the major issues, set out a timetable, and provided an email linkfor responses and suggestions 
[http://www.mnsu.edu/dept/ncafocus/ncahome.htmO. 

While we did not receive a great deal of commentary through the email link or at the open meetings, the comments 
and feedback that we did get through all of the meetings were invaluable. They helped us understand the specific 
achievements of the campus, the obstacles that it had 
faced, and the campus opinion about those achieve- 
ments. The information and comments we collected 
gave us a solid foundation with which to begin the first 
draft of the Self-Study Report. 



Circulating Drafts of the Focused Visit 
Self-Study 

We began writing the Self-Study Report by the middle of 
summer 1 999. We knew that we could not afford to wait 
until the fall because we wanted to show a completed 
draft to our NCA contact by early November and to have 
had time for the campus to respond to at least one 
complete draft before that. We worked collaboratively, 
with each of us writing different sections of the report 
and the other reviewing, editing, and revising what the 
initial author had written. By the end of the summer, we 
had a complete draft ready to show to the campus. 

During September and early October, we solicited com- 
mentary on the draft. We sent paper copies of the entire 
draft to each of the key individuals and groups we had 
identified and to each department and support office on 
campus. We also posted the complete draft on our web 
site, with links for email feedback. We again scheduled 
meetings with key individuals and committees and asked the deans, other administrators, and other leaders to discuss 
the draft with their units. We also sought to publicize widely these efforts to the campus and larger community. We 
held interviews with editors and reporters from campus publications and from the local city newspaper. In the 
meantime, the NCA Advisory Committee was also meeting to review and discuss the drafts. Thus, we continued to 
educate everyone on campus about the purpose of the Focused Visit and how the university was responding to the 
issues while preparing for the visit itself. 

We used the new comments we were receiving as a basis for further revision of the Self-Study Report. Many groups 
had suggestions for clarifications and improvements, which we appreciated. We were sometimes disappointed by the 
comments that were the least critical. While it was heartening to be told that we had done a “great job,” comments 
that targeted specific elements of the report for revision [or for praise!) were far more helpful to our rewriting efforts. 



Chronology for Focused Visit Preparations at 
Minnesota State University, Mankato 


Fall 1999 


Co-chairs appointed 


Spring 1999 


Focused Visit Advisory Committee 
begins to meet 

Fact-finding meetings, open dis- 
cussions, written communications 
about Focused Visit issues 


Summer 1999 


Co-authors write complete draft of 
campus Self-Study Report 


Fall 1999 


Self-Study draft is circulated for 
discussion and comments 

Self-study draft is revised and sub- 
mitted to NCA contact for com- 
ments and feedback 


December 1999 


Final Self-Study Report is submitted to 
NCA and Focused Visit team 


January 2000 


Final self-study is distributed on 
campus in preparation for visit by 
Focused Visit team 


February 21-22, 2000 


Focused Visit occurs 


March 2000 


Advisory Committee meets to as- 
sess the visit 



Preparing and Distributing the Final Version of the Self-Study 

As we received comments on the draft, we continued to revise the report and to present our revisions to the Advisory 
Committee. By early November, we submitted a complete report to the president and senior vice president, who then 
sent it on to our NCA staff liaison. In December, we took the minor comments from the NCA staff and did our final 
revision and polishing of the report. Soon afterwards, the final version was submitted to NCA and the Focused Visit 
team in preparation for their February visit. 

The final version of the report was also distributed in hard copy to all of the identified constituencies and posted on 
the university web site [http://www.mnsu.edu/dept/ncafocus/report.html) so the campus community would know 
how we had used their input and so that everyone could see the final document that was given to the Focused Visit 
Team. We encouraged the deans and other administrators to have a final discussion of the report with their units prior 
to the visit. We worked with the bargaining units and key campus committees to be sure that their members were 
well aware of visit details. Through email announcements and the web site we continued to update the campus on 
the preparations for the visit. 
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Preparing for the Focused Visit 

Even as we were writing the final drafts of the Self-Study Report, we were also preparing for the actual visit by the 
Focused Visit team with the close participation of the president and senior vice president. The assistant vice president 
for undergraduate education and other members of the academic affairs staff gathered needed documentation, 
collated and assembled it in binders, and prepared a Resource Room where the campus community could examine 
the background documents and where the Focused VisitTeam could meet and work. The president’s office arranged 
for housing and set up the schedule for the visit itself. 



Outcome and Conclusions 

The actual Focused Visit went very smoothly. The report helped the team members understand the issues of concern 
and the ways in which the campus had identified and worked to deal with those issues. The campus was well-prepared 
for the visit, and key individuals and groups were ready to discuss the campus’s achievements in specific and concrete 
ways. The open meetings with faculty end with ether campus groups were weii-attended. The evaluation team was 
able to report that Minnesota State University had met or was meeting the concerns for the Focused Visit and to 
recommend no further Focused Visits or other reports before the next full reaccreditation visit. 

We were pleased that the visit had been so successful, and we were pleased at how actively the campus had 
participated in preparations for the self-study and visit. In a final, post-visit meeting of the Advisory Committee, the 
members commended the process and the opportunities for campus input and feedback. Though some committee 
members still wished that there had been even greater participation from those outside of academic affairs units, all 
agreed that the visit had been a success because of the widespread knowledge of the visit and the frequent 
communications. Of course, all of these efforts would have been for naught if the university had not made substantial 
progress in the Focused Visit areas! 



Donald F. Larsson is Professor of English and Director of the Humanities Program at Minnesota State University in 
Mankato. 

Susan Coultrap-McQuin is Dean of the College of Social and Behavioral Sciences at Minnesota State University in 
Mankato. 
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Preparation for a 

Commission-Mandated Focused Visit 



Patricia Donohue 
John Cosgrove 
Sally Souder 
Larry McDoniel 
Donna Spaulding 



Introduction 

In 1998 an NCA Evaluation Team recommended that St. Louis Community College be granted continued accredited 
status with the next comprehensive evaluation in ten years, 2007-2008, and a focused visit scheduled for 
January 29-30, 2001. The focused visit was arranged for the purpose of reviewing three areas of concern: district 
governance and leadership, human resources, and assessment. 

Between 1 966 and the 1 998 visit, the college had been visited by NCA four times. Since 1 988 the college has been 
accredited as a single institution, not as individual campuses. 



Background 

St. Louis Community College is a public coeducational college supported by local taxes, state funds, and student fees. 
Created by area voters in 1962, the college offers freshman- and sophomore-level career, college transfer, 
developmental, and continuing education programs at its three campuses, four education centers, and numerous 
other locations throughout St. Louis City and County. More than 1 30,000 students enroll each year in credit and non- 
credit courses. The college also serves the business community with programs offering counseling, consulting, and 
training services. 



Leadership Needed 

In order to prepare for a successful focused visit, it is imperative to have strong leadership. In the case of SLCC, this 
began with Vice Chancellor for Education Patricia C. Donohue. Dr. Donohue, as the chief academic officer for the 
district, was named the focused visit coordinator. Throughout this time, she has worked in conjunction with John 
Cosgrove, who is the Director of Institutional Research and Planning. This office has provided the research for the 
Focused Visit Report 

Faculty members have been involved in leading the preparation for the visit as well. The faculty member who served 
as a campus NCA Coordinator and district Editor for the 1998 Self-Study Report transitioned to the position of 
Administrative Intern to the Vice Chancellor for Education. Forest Park Associate Professor of English Sally Souder 
provided continuity of leadership for NCA issues in this one-year released-time position for a faculty member who 
assists Dr. Donohue. Associate Professor of Speech and Theatre Donna Spaulding transitioned into this position for 
the remainder of the preparation forthe visit. Their responsibilities included regular monitoring of progress, the writing 
and printing of the Report, organizing materials for the Resource Room, keeping the college community informed, 
and making preparations forthe team's visit. 
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In the SLCC district, the Assessment Coordinator is a faculty member who has been released from regular faculty 
duties for a period of three years. As assessment was one of the areas cited for the focused visit, this person's 
leadership became very important. At the beginning of the three-year period, Dr. Richard Baker, Professor of History 
at the Meramec campus, held this position. Later, Dr. Larry McDoniel, Professor of English at Meramec, took over. 
The Assessment Coordinator leads the District Assessment Council, coordinates the campus assessment councils, 
and coordinates and facilitates the corps of assessment resource persons in departments across the district. Both 
Baker and McDoniel have played important roles in the preparation for the focused visit. 

Beyond these leaders, the effort could not succeed without the support of the Board of Trustees and the College 
Leadership Team. The six-member Board of Trustees has been active in attending NCA conferences, offering advice, 
and meeting with the visiting team to demonstrate ongoing support. The Leadership Team includes the Chancellor 
of the college, Dr. Henry Shannon; Dr. Donohue; Vice Chancellor for Finance and Business Services Carla Chance; 
Florissant Valley President Gustavo Valadez Ortiz; Acting Forest Park President Ronald Smith; Meramec President E. 
Lynn Suydam; Director of Human Resources Ronald Nicoletti; and General Counsel Tina Odo. Members of this group 
have offered input to the report and have served as members of the Focused Visit Self-Study Steering Team. 



Maintaining Momentum 

Everyone reading this report knows the work involved to prepare for a comprehensive NCA visit. It is no small task. 
Many, if not most, members of the college community have been involved in one or more ways. Now imagine that 
the visit is over, and everyone wants to breathe a sigh of relief, but now you are told that a team will be back in three 
years to reexamine some specific issues. 

If you can imagine this scenario, you have an idea of the reaction and feelings that must be overcome to prepare for 
the focused visit. There is no time to “let down” because the team will be here in a few short years, and there is much 
work to be done in the interim. 

SLCC used several approaches to maintain the momentum toward the focused visit. One strategy was to immediately 
begin work in addressing the three issues. The NCA Coordinator for the 1998 self-study and key members of his team 
drafted a structure for addressing the three focus areas shortly after the preliminary review of the Team Report. This 
provided continuity and tapped the expertise of those most knowledgeable from the self-study process. This 
framework was used to immediately engage key staff with responsibilities for focus areas, who converted it to an 
action plan and schedule. 

Since one of our issues was assessment, the Coordinator of Assessment had a key role in focusing college efforts 
to align with NCA concerns. Of the three issues, assessment was the one that affected absolutely everyone in the 
college. Because of the visibility of the coordinator and the building and execution of the Assessment Plan, everyone 
was constantly reminded that this work was ongoing. 

Another approach was the execution of an extensive communications plan for the focused visit. We built a web page 
exclusively for the visit. It included the most up-to-date information available and included a direct link to the 
Administrative Intern, who was responsible for responding to any questions. It also included a direct link to the NCA 
home page for individuals who wanted to pursue information there. A document entitled “Progress Update” was 
created by the Administrative Intern as a summary of the action plan. The “Progress Update” was revised on a regular 
basis and distributed in a variety of ways. It was published at least once every semester in the site newsletters, included 
on the web page, and distributed freely at a variety of meetings. 

The Administrative Intern was responsible for giving presentations and handing out copies of the “Progress Update” 
at meetings of the Board of Trustees, Leadership Team, Presidents’ Cabinets, Vice Chancellor staff, Deans’ Council, 
governance councils, student government associations, and many more. 

An NCA Strategic Steering Committee was named to focus attention on and begin to plan for the next comprehensive 
visit in 2008. This committee is composed of representatives from administration, faculty, faculty union, governance 
councils, and strategic planning. The committee has created a planning document and serves as a think tank to assure 
we are addressing all important and necessary issues prior to the 2008 visit. 

All these approaches came out of the office of the Vice Chancellor for Education. Dr. Patricia Donohue was the 
Focused Visit Coordinator and was the inspiration and driving force behind all these efforts. An amazing amount of 
work was accomplished during the three years from 1 998 to 2001 . We feel there has been no loss of momentum since 
the comprehensive visit. 
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Preparing the Document 

Before writing the first draft, the Administrative Intern read many documents and met with various councils and 
administrative groups. She worked closely with the Chancellor and the Vice Chancellor for Education in preparing 
the first draft of the chapter on governance. When the first draft was finished, the Chancellor and the Vice Chancellor 
examined it, as did the Leadership Team, which includes the three campus presidents. After receiving their 
suggestions, she wrote a second draft. 

That format held true for the chapters on human resources and assessment. The intern met with the Human Resources 
Director numerous times, as well as with the Assessment Coordinator and the Institutional Research and Planning 
Director. After she wrote a first draft, these officers examined it; the intern made corrections and then sent the 
chapters first to the Vice Chancellor for Education and then to the Leadership Team for review. The Deans’ Council 
also examined and commented on the chapter drafts. Then the intern wrote a second draft, incorporating their 
suggestions. 

SLCC submitted the second draft to SLCC's NCA staff liaison, who read it and made suggestions. The process 
continued in this way with new information being added and reviewed by SLCC officers and administrators. Members 
of the SLCC Board of Trustees also received copies of the second draft. 

This process worked well for a focused visit self-study limited to three topics. It was quite a departure from the process 
that was used to write the comprehensive self-study for the visit in 1998, when many more members of the college 
community were actively involved in writing and critiquing the document. 



Assessment as an Ongoing Issue 

In preparing for and following the 1998 visit, administrative support for assessment was demonstrated by the 
appointment of the full-time Assessment Coordinator who reports directly to the Vice Chancellor for Education, by 
the hiring of an Assessment Associate, and by financial support for assessment. Faculty and staff support was 
demonstrated by participation in a variety of assessment activities, detailed below. 

□ Revision of the Assessment Plan 

After the NCA visit in March 1998, the Summer (1998) Task Force on Assessment revised the 1996 assessment 
plan and created a five-year strategy for full implementation of assessment at St. Louis Community College. This 
plan was submitted to the college faculty and to the College Academic Council, and it was amended through joint 
work by this task force and NEA representatives. 

Implemented in January 1999, the five-year strategy includes provisions for: 

o Classroom assessment 
o Course assessment 
o Program assessment 
o College services assessment 
o Assessment of assessment 
o Organization and supervision of assessment. 

In each case, the proposed assessment involved students, faculty, staff, and administration in kinds and amounts 
of assessment rarely attempted before. The “culture of assessment" took a quantum leap. 

□ Assessment for NCA and Others 

As SLCC approached its January 2001 focused visit, the existing assessment committees— a district-wide 
committee plus one at each of the three campuses— as well as the newly created assessment resource persons 
(ARPs) from each academic department in addition to the assessment liaisons to the various college services 
worked hard to implement the assessment procedures recently put into play. Because so many of our assessment 
policies and procedures were newly devised, considerable review and revision was necessary to refine what was 
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being attempted. Anew reporting mechanism called CARIS was commissioned and created. This intranet prog ram 
allowed those involved in assessment to submit and track assessment reports immediately, thus eliminating the 
perpetual paper shuffle and its associated hazards. 

Moreover, SLCC became even more attentive to assessment issues when presented with a state-wide mandate 
to describe and confirm students’ competency in general education. Consequently what was gained in meeting 
the demands of an NCA focused visit was soon paying dividends for those responding to the Missouri’s general 
education project 2000-2001, for assessment is a significant issue in the general education “draft” to be posted 
on a state-wide web site by mid-May 2001. 



Focused Visit Issues 

□ District Governance and Leadership 

In order to write this portion of report, we had to begin with observation and reflection. Several of the issues the 
Visiting Team mentioned had to do with the specific personnel who were here in 1998. Since then, some of the 
administrators and Board of Trustees members have changed. These changes helped eliminate some of the issues 
that had been cited. Another team concern was the internal governance process. Again, changes were made and 
reported for the focused visit. An examination of the minutes of the council meetings provided evidence for the 
report. 

□ Human Resources 

This part of the report was written in conjunction with HR personnel. Most of the team concerns in this area were 
specific and were best answered by members of the HR staff. We requested copies of new and updated documents, 
studies, and plans from their office. We conducted interviews with them and enlisted their aid in writing this section. 

□ Assessment 

This portion of the report may have been the easiest to write because there was so much activity in the area of 
assessment. Virtually the entire college was involved in assessment activities in a very structured way. As 
mentioned earlier, the Assessment Coordinator led us through the steps of the assessment plan. The many 
activities, reports, council meetings, and newsletters provided the information forthis partof the report. The college 
has truly developed a culture of assessment, and the report provides extensive evidence of this. 



Conclusion 

SLCC has worked hard to assure that we have a successful NCA focused visit in January 2001. We believe we have 
made substantial progress on each of the issues. It is difficult to give closure to this paper because we have not yet 
experienced the focused visit. When we give the presentation at the NCA Meeting, there will be new information to 
share with our audience. 



Patricia Donohue is Vice Chancellor for Education at Saint Louis Community College in Missouri. 

John Cosgrove is Director of Institutional Research and Planning at Saint Louis Community College in Missouri. 
Sally Souder is Associate Professor of English at Saint Louis Community College in Missouri. 

Larry McDoniel is Assessment Coordinator at Saint Louis Community College in Missouri. 

Donna Spaulding is Administrative Intern at Saint Louis Community College in Missouri. 
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Why Are We Doing All This Work? 
The Real Benefits of Pursuing 
Regional Accreditation 



Terry Allcorn 
Ronald Oakes 

When we entered into the accreditation process several years ago, we knew it was important to have regional 
accreditation to ease the transfer of credit to other regionally accredited institutions. While this is true, the 
accreditation process is a lot of work if we primarily sought accreditation to make it easier for our learners to leave. 
A few of us, though, had a difficult time articulating what regional accreditation implied for a college that was already 
accredited with a national accrediting association. By going through the self-study process and achieving candidacy 
status, we discovered some very real benefits to affiliation with The Higher Learning Commission that have an even 
more profound impact on our institution. 



Background 

Founded in 1956, Saint Louis Christian College is a private, church-related, nonprofit coeducational baccalaureate 
institution of higher education in the tradition of the Bible college movement Since 1 977, SLCC has been accredited 
with the Accrediting Association of Bible Colleges (AABC3, an association recognized by the U.S. Department of 
Education as the institutional accrediting agency on the national level for Bible colleges. Through the processes of 
revising our mission statement (1989-19933 and developing an objective-driven curriculum (1993-19963, we began 
to posture ourselves for the institutional challenges associated with the pursuit of regional accreditation. 

The mission of Saint Louis Christian College is to provide Bible-centered higher education for men and women 
preparing for Christian service. As a small institution with 200 learners enrolled in four degree programs (BA, BS, AA, 
AAS3, including a nontraditional adultdegree-compietion program, we had been reaffirmed and granted anotherten- 
year accreditation status with AABC in 1998. There were a number of employees who questioned the wisdom of 
pursuing regional accreditation. 

As a result, several barriers were raised in the form of questions that demanded an institutional response: Why is 
regional accreditation necessary? How will we be able to develop campus-wide ownership of this process? How can 
we overcomethe perception thatassociation with NCA would somehow diminish our commitment to Christian higher 
education? How would NCA evaluate our Bible-based curriculum? Will NCA’s emphasis on assessment of student 
learning force us to violate our own mission and core values? 



Concurrent with the AABC self-study, the Trustees in November 1997 approved seeking affiliation with NCA. The 
Preliminary Information Form was sentto NCA three months after being reaffirmed with AABC. Later in 1 998, the NCA 
staff analysis of our PIF approved nineteen of the twenty-four GIRs, which represent the entry-level threshold for 
association with the Commission. Not only did we have to respond to the questions, but now we had to develop an 
action plan to address the five remaining GIRs. 




In an effort to address those concerns and to keep moving forward in the pursuit of candidacy status, we developed 
some strategies that will assist you in the process as well. 
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Strategies That Benefited Us 

□ We Sought Extensive External Counsel and Advice 

Off campus: We investigated what others were doing in key areas of institutional effectiveness. 

1. We sent representatives to an assessment conference at Alverno College to learn from their expertise. 

2. We sent six representatives to last year’s NCA Annual Meeting. 

3. We sent a representative to speak with a college that was denied candidacy status. They were kind 
enough to allow us to view their materials and to give us some insights into our own situation. 

4. Every time we send someone to a conference, he or she brings back important information to us. We are 
a better college because of it. 

On campus: We brought in people who could provide experienced counsel. 

1. A consultant-evaluator who serves as an administrator from a college with a similar mission statement 
and purpose addressed the entire college community. He explained the benefits of regional accreditation 
and helped alleviate some concerns that had been expressed. We purposefully targeted everyone, not 
just the faculty and not just those directly involved in the process. 

2. Another administrator helped us address one of our areas of concern: that of assessment. 

3. We invited our NCA staff liaison to visit our campus and to provide additional constructive suggestions. 

□ We Developed Strategic Response Plans to Address Institutional Weaknesses 

1. For example, we targeted specific curriculum issues that the process had brought to light. We were 
particularly concerned about our general education curriculum. Much of what we do is fairly specialized 
in the other two divisions, Biblical and professional. We knewthatthe one thing we would have in common 
with every evaluator was general education. 

2. We completed a comparative program and curriculum analysis of both NCA-affiliated institutions and our 
sister Bible colleges that were regionally accredited. We evaluated total required hours in the general and 
professional divisions as well as the Biblical division. We identified specific courses required in those 
institutions. All of this was to determine common practice in institutions of higher education. 

3. As a result, we adjusted our general education curriculum and added a full-time English professor. 

4. The faculty made a decision not to reduce our Bible requirements below39 hours for the four-year degree 
programs. We felt this was in line with our mission and purpose. We have also recently begun considering 
an adjustment to our curriculum to meet the new Missouri guidelines. This close scrutiny of our 
curriculum is a result of our association with the NCA. 

5. Every one of these actions improved our institution and the education we offer to our learners. During 
this period, our institution made the key transition from making adjustments to receive accreditation to 
making adjustments to improve the quality of our institution. This was more than a change in semantics; 
it was a change of motivation. Our strong belief is that withoutthis transition, while we might have received 
candidacy status, we would have not understood the real fruits of all our labors. 

□ We Sought Internal Counsel and Advice 

1. We surveyed our learners to get an accurate view of their perceptions of our program and their general 
level of satisfaction. 

2. We held meetings where all campus personnel were able to contribute their ideas and opinions 
concerning where our priorities should be. 

3. We used interviews and questionnaires to reach learners, faculty, staff, administration, trustees, and alumni. 

4. All of this gave us a fuller understanding of both how we perceived ourselves and where we should 
concentrate our efforts to improve our programs. 
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□ We Gave Extensive Communication to the College Community 

1 . The President's office sent internal newsletters to the campus community. The Self-Study Committee also 
circulated information. 

2. Our constituency was kept informed by the college newsletter. The learners were kept informed through 
a newsletter published by Student Services. 

3. When our self-study was completed, we created an executive summary to provide to the campus 
community and constituents. This summary has been very well received. It allowed us to briefly show the 
results of our self-study without the huge amount of information in the principal documents. 

□ We Conducted a Mock Evaluation Team Site Visit 

1. We scheduled a mock team visit about one month prior to the actual site visit. The team was made up 
of in-state consultant-evaluators with site visit team experience. This recommendation by NCA was one 
of our best strategies. We were able to allow the campus community to experience what a real visit might 
be like and benefited from the counsel of experienced consultants. 

2. The consultants helped us identify weaknesses in areas such as strategic planning, identifying the 
process to respond to perceived institutional challenges, and the compilation of documentation for our 
Resource Room. The consultants looked at our preparation for the actual site visit to help us put the 
finishing touches on our presentation to the actual site consultants. 

We would like to be able to say that we would have done all of the above whether we would were applying for NCA 
affiliation or not. As an institution, we are concerned about offering a high-quality education to our learners. However, 
our association with NCA has provided us with the resources to offer a program of higher quality than we could have 
done on our own. 



The Strategies Produced Results That Benefited Us 

Ourcandidacystatuswith NCA provides us with the credentials togive both our constituents and those notassociated 
with the college confidence that we are a reputable institution of higher learning. However, affiliation with the 
Commission has had a profound impact on our institution in more substantial manners: 

1. Helped to strengthen our self-identity. It helped us identify and solidify who we are. We were asked to 
identify our role in higher education, identify our core values, and determine how we can better fulfill our 
mission. 

2. Increased the quality of our curriculum. It raised the quality of our curriculum, specifically our general 
education curriculum, to new levels of excellence and made it compatible with what most see today as a well- 
balanced program. 

3. Focused our attention on student learning and assessment. It has raised our awareness of the need to 
assess all that we do as an institution and to compare that assessment to national norms and trends. 

4. Increased institutional ownership. It raised the level of cooperation and community of all parties involved 
in the life of the institution. It has brought a new level of energy to the campus. We chose to use our size to 
our advantage. Almost everyone on campus played some role in preparing for our affiliation with the 
Commission and shares in the satisfaction of having done so. 

5. Revitalized our trustees and alumni. They saw something happening on campus to be excited about. They 
are beginning to catch a glimpse of what the institution can be and are taking more ownership in the life of 
the college. 

6. Heightened our concern to provide a fully credentialed faculty. We now do a better job completing the 
professional portfolios of our full-time and part-time instructors. We also take a more critical look at our 
adjunct hiring practices. 

7. Clarified institutional principles, policies, and procedures. We are more organized. We developed 
manuals for every area of the institution. The faculty had benefited from a manual for some time. However, 
it needed to be revised and updated. The administration, staff, adjunct faculty, and trustees had functioned 
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for more than 40 years without a manual. Lines of authority and procedures are now clearly spelled out We 
are more structured. We now document the process as well as the decision. It raised our awareness of the 
fact that the process by which we make a decision is just as important as the decision itself. 

8. Demonstrated the determination of the institution. It showed us just how far we could push ourselves. 
Our aspiration to favorably compare ourselves with the best institutions of higher education took our best 
efforts, both individually and corporately. 



Conclusion 

Our contact with The Higher Learning Commission has always been cordial and collegial. It soon became apparent 
to us that, while the Commission expected the highest quality in our programs, their evaluations were intended to help 
us improve the quality of our educational programs and not to criticize them. The Higher Learning Commission never 
asked us to make any adjustments to our institution that went against our mission statement. In fact, they would have 
counseled us against contradicting our mission statement or our institutional core values. 

We benefited from both the strategies to address the problems Cthe process] and the actual results of the strategies 
Cinstitutional improvements]. We are a much better college because we decided to affiliate with the NCA. While we 
would have been greatly disappointed if we had not gained candidacy status, it would not have removed all of the 
progress our institution made as we went through the process. 



Terry Allcorn is Chair of General Education at Saint Louis Christian College in Florissant, Missouri. 
Ronald Oakes is Academic Dean at Saint Louis Christian College in Florissant, Missouri. 
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Bridging the Gap to 
Accreditation and Beyond: 

A Case Study of How the 
Self-Study Process Shaped an 
Institution Moving from 
No Affiliation to Initial Accreditation 



Myleea Hill 
Christy Huddleston 



Background 

Regional accreditation was part of the original vision of Crowley's Ridge College when it was founded in 1964. While 
various obstacles and unforeseen challenges caused an extended delay, CRC never lost sight of its goal. A 1992 
federal ruling putting CRC students’ eligibility for federal financial assistance in jeopardy provided the impetus for 
CRC to begin the process of seeking accreditation with the North Central Association of Colleges and Schools in 
earnest. CRC received candidacy status in 1996 and continued candidacy in 1998. Granted initial accreditation in 
August 2000, Crowley’s Ridge College is an example of the impact the self-study and accreditation process can have 
in shaping an institution. In just over five years, CRC bridged the gap from non-affiliation to initial accreditation. 

The NCA staff response to the CRC Preliminary Information Form in 1 994 laid out a road map for needed improvements 
in the college’s educational programs and operating structure. Since submitting the PIF, CRC has undergone a steep 
learning curve to meet General Institutional Requirements and Criteria for Accreditation. Benchmarking and 
increased understanding of “common practices” in higher education precipitated organizational and cultural 
changes as the private two-year college made the journey to accreditation. Information gleaned during NCA Annual 
Meetings prepared the college to meet challenges and recognize opportunities during the mid-1 990s. The lessons 
learned from the self-study process have been beneficial as the college builds on accreditation and prepares to focus 
on success rather than survival. 



"BASSic" Building Blocks 

CRC’s experience provides a case study of the opportunities and obstacles facing institutions in the self-study 
process. It also demonstrates how the road to accreditation shapes an institution and leads to significant and systemic 
organizational changes in budgeting, assessment, shared governance, and strategic planning. These four key issues 
served as signposts as CRC traveled the road from non-affiliation to an accredited institution. An acrostic of BASS 
was used to emphasize to the college community necessary modifications. The progress in budgeting, assessment, 
strategic planning, and shared-governance initiatives cut to the heart of how CRC operates and makes decisions. 
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0 B = Budgeting. The 1994 PIF and 1995 Evaluation Team both commented on the highly centralized 
budgeting processes. During the 1998 team visit, the process was improving but not fully functioning. 
By the 2000 visit, the self-study found and Evaluation Team confirmed a budgeting process that was linked 
to shared-governance and strategic planning, with budgeting managers exercising control over their 
divisions. 

Strategic planning, shared-governance procedures, and board directives have an impact on the way the 
college derives and spends money. Through budgetary processes established during the extended self-study 
process, the college improved its ability to project income and track expenses. Budgets are established 
through strategic planning, which sets priorities necessary to reach agreed-upon goals. Line-item monthly 
budget reports notify department budget managers of available funds. The collaborative process has resulted 
in a decentralized system in which budget managers have both more responsibility and the authority to make 
fiscal decisions in keeping with established priorities. 

As an institution of higher learning, CRC places emphasis on support of instruction. The restructuring of 
budget processes placed priority on assuring that necessary learning equipment be made available. The 
budgetary cycle allows for “phases” in which urgent-particularlv academic-items may be purchased at the 
beginning of the fiscal year. Approval of requested non-essential items may be contingent on available funds 
and the college’s overall financial condition based on mid-year checkpoints. The system has allowed the 
college to balance a “no-deficit spending” directive by the board with the need to maintain proper learning 
support. 

0 A = Assessment. Formal assessment of academic achievement has led to significant and substantial 
improvements in CRC’s academic program. Such changes include the establishment of a developmental 
program, restructuring courses, and outlining measurable goals. The use of assessment has led to 
documentation of student learning as well as recommendations for providing programs in support of the 
college’s open admissions policy. With oversight by faculty and support from the administration and board, 
the assessment program has been a key tool in quality improvements to educational programs. One of the 
most important utilizations of assessment data is through strategic planning in establishing budgetary and 
funding priorities for instructional materials and professional development for instructors. 

Where in the past CRC relied on informal and anecdotal evidence, formal assessment methods nowdocument 
educational outcomes and point to areas of improvement. An unanticipated but welcome result of formalized 
assessment has been its effect on campus morale and the college's overall image. Alumni reports and 
proficiency tests demonstrated that CRC students scored in a comparable range with regional and national 
students. These findings energized the campus community as it began not to look just for ways to prove that 
the college offered a quality education but to improve the quality of education that is being provided. 

0 s = Shared governance. Arguably, the single biggest contributor to progress at CRC in recent years has 
been shared governance because of the positive effect it has had on communication and information flow 
between and among students, faculty, staff, administrators, and the board of trustees. Through shared 
governance, the college has moved from a centralized bottle neck to an increasingly empowered organization 
with personnel having the authority to make decisions and implement policies to improve the operation of the 
college. Shared governance has boosted morale and enabled employees to feel a sense of ownership in the 
college. 

Shared governance has functioned both through administratively appointed committees and an autonomous 
faculty senate. Where appointed committees include a cross-section of students, faculty, administration, and 
staff members to accomplish designated agenda items, the faculty senate incorporatesfull-time and part-time 
faculty members [excluding administrators] and distributes minutes directly to the board of trustees and other 
campus constituencies. Established in 1 995, the faculty senate has a tradition of discussing issues impacting 
the campus learning environment. The board has credited faculty senate minutes with providing vital 
information as it made policy necessary to accomplish CRC's mission. 

Appointed shared-governance committees have also been effective in maximizing the varied experiences and 
expertise of CRC employees. The collaboration of shared governance enhanced morale, as “we” worked 
together to solve problems rather than talking about something “they” should do. However, shared 
governance may have fallen prey to the “too much of a good thing” syndrome. Over the course of five years, 
the shared governance system ballooned to up to 1 6 committees, which proved to be overly cumbersome for 
a college of less than 1 0 full-time faculty members. Through the 2000 self-study cycle, CRC recognized the 
need to restructure and has since consolidated committees. Shared governance continues to be a powerful 
force on campus as administrators, faculty, and staff work side by side and employees provide leadership to 
teams across organizational chart lines. 
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0 S = Strategic Planning. Making and implementing long-range plans has been perhaps the biggest 
challenge in an institution that has traditionally focused more on survival than planning for the future. Through 
an increasingly comprehensive five-year strategic plan, the college developed plans and methodologies for 
implementation. Initiated in 1995, a single planning team with representation from the community, board, 
administration, alumni, faculty, and staff laid the groundwork for strategic planning at CRC. In 1997, a board 
resolution shifted the strategic planning process to a participative, mission-driven process including short- 
and long-term plans of up to five years. The college mission and goals were integrated with specific planning 
goals to ensure that planning included a student focus. A scenario planning model helped foresee possible 
conditions, but was viewed by the Evaluation Team as lacking in specificity. In 1999, strategic planning was 
further revised to broaden campus participation and allow for comprehensive inputs and specific targets in 
enrollment and fundraising. Appointing planning teams and plan managers effectively empowered employ- 
ees to take steps and make improvements in their area and also increased a culture of accountability. By 
involving a variety of stakeholders in planning, CRC has taken advantage of the insights of personnel who are 
responsible for executing plans. Because budgeting is coordinated through strategic planning, it remains a 
vital and evolving process for the college. Many of the self-study recommendations and responses to the 
visiting team concerns are channeled through strategic planning teams, so it continues to play an integral role 
in the progress of the college. 

While the methods mentioned above undeniably provided the structure CRC needed to meet NCA criteria, it is also true 
that the processes are hollow without people to make them work. The personnel at CRC evolved both in number and 
outlook as long-time and new employees rose to the challenge and overcame obstacles. Ultimately, the newly instituted 
systems combined with a dedicated workforce to reach a “point of no return” in progress toward accreditation. As the 
systems became more and more ingrained into the college culture, personnel became more confident in their abilities 
to make and implement decisions. A common phrase used to explain both setbacks and successes has been 
“communication." As the saying goes, “Talk is cheap, but communication is priceless.” Once personnel reached a point 
of realizing that structures were in place, ideas turned into action and action turned into results. 

Step by Step 

CRC ran into its share of roadblocks as it pursued accreditation, but typically even setbacks led to improvements. 
Having begun addressing items from the PI F submitted in 1994, the college took a much anticipated first step in 1996 
when it received candidacy status. In 1998, CRC decided that it could benefit from being evaluated next to the higher 
standards of initial accreditation even though it could remain in candidacy two more years. When a visiting team listed 
17 concerns in recommending continued candidacy, the college viewed it as a wake-up call. 

Renewed efforts, along with encouragement from the 1 998 team that CRC had “made steady progress,” brought the 
college community to the 1999-2000 self-study cycle. While conducting three self-studies in five years was 
challenging, it also helped the college measure its progress and recognize accomplishments in organization and 
education. At the kick-off meeting for the 1999-2000 self-study hosted by the faculty senate, the Self-Study 
Coordinator encouraged employees and board members to “build our case” for accreditation. Looking back on five 
years, documented progress was evident not just in the processes noted above but also in facts and figures such as 
enrollment and fundraising. Once completed, the 2000 Self-Study Report was distributed to the college community 
with the admonition to "believe we belong.” The saying was largely based on assessment data that showed the 
success of CRC graduates in comparison to students across the state and nation. Finally, posters, memos, and public 
announcements encouraged students, employees, alumni, and the community to “tell our story” to the Evaluation 
Team. Team members complimented the show of community support and especially the attendance at an open 
student meeting, (forty-four out of 153 students attended.] While the connection between processes and people may 
not seem obvious, at CRC they worked together to build an atmosphere in which people truly believe their input makes 
a difference. 

Onward and Upward 

CRC is understandably pleased with the progress noted in the self-study and verified by the visiting team, but the 
desire is for accreditation notto be an end in itself, but rather a springboard to a brighterfuture. The Self-Study Report, 
visiting team report, and institutional response outline focus items, concerns, and recommendations. Obstacles, 
opportunities, and challenges have developed as CRC makes the transition from the intensity of the self-study to trying 
to institute sustained improvements. In the months following obtaining accreditation, plans have met with both 
setbacks and successes. While some factors-such as the resignation of the chief fundraising officer, the retirement 
of the director of research and planning, and the announced retirement of the president-were unanticipated, a more 
formalized structure could have helped maintain momentum. 
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The NCA Handbook of Accreditation explains the importance of establishing methods for addressing the findings of 
the self-study. “As part of the formal self-study process the Steering Committee or some other new or existing group 
needs to be charged by the Executive Officer, after the Evaluation Team leaves, to track the recommertdationsthe self- 
study produced; the recommendations need to be assigned formally to those individuals and groups in the institution 
whose job it is to deal with the areas concerned.” [NCA Handbook of Accreditation, Second Edition, p. 77). In previous 
self-study cycles, the steering committee remained intact because a new Self-Study Report was due to be produced 
in two years. After receiving initial accreditation in 2000 and the subsequent turnover in personnel, the steering 
committee did not continue this role. A Self-Study Coordinator’s compilation was distributed to faculty members and 
planning teams. Although items are being addressed, it has become apparent that a transition team could have been 
advantageous in building on momentum from the self-study process. So, while there is much to be learned from the 
case study of CRC’s accomplishments, other institutions are also invited to improve on CRC's methods by formally 
assigning a group to track recommendations. 



Summary 

fviany systemic organizationai changes in budgeting, assessment, shared governance, and strategic planning 
developed as CRC bridged the gap to accreditation. The accreditation journey didn’t end with the self-study, the 
Evaluation Team, or even the Review Committee. CRC has learned that once an institution crosses the bridge to 
accreditation there are new roads to travel. Just like getting to accreditation, having-and following-a road map is 
a good idea. 
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